Florida Assessments for Instruction in Reading Third Grade - Twelfth Grade

Transcription

1 Florida Assessments for Instruction in Reading Third Grade - Twelfth Grade Administration Manual

2 This product is protected by the copyright laws of the United States. Materials may not be copied, reproduced, republished, uploaded, posted, transmitted, distributed, or modified without the written consent of the Florida Department of Education, Tallahassee, Florida.

3 Contact Information Florida Department of Education Just Read, Florida! 325 W. Gaines Street, Suite 1548 Tallahassee, FL Florida Center for Reading Research Florida State University 2010 Levy Avenue, Suite 100 Tallahassee, FL (850)

4 This page intentionally left blank

5 Table of Contents Overview and Administration Guidelines...1 Grades 3-12 Assessments Overview...3 Assessment System Conceptual Model...7 Technical Requirements...11 Assessment Timeline...13 Roster Confirmation Prior to Testing...15 Roster Confirmation Procedure...16 Web-based Assessment Module (WAM) Manager Functions...19 Web-based Assessment Module (WAM) Student Functions...25 Implementation Tips for Schools...31 Broad Screen/ Progress Monitoring Tool (BS/PMT)...33 Assessment Description and Purpose...35 Reading Comprehension Task Administration...36 Targeted Diagnostic Inventory (TDI)...39 Assessment Description and Purpose...41 Maze Task Administration...42 Word Analysis Task Administration...45 Ongoing Progress Monitoring (OPM)...47 OPM Reading Comprehension...49 OPM Maze...49 OPM Oral Reading Fluency...50 Informal Diagnostic/Progress Assessment Tool Kit...51 Diagnostic Tool Kit Description and Directions for Use...53 Instructional Implications...63 Using Assessment to Guide Instruction...64 Appendix...69 Accommodations, Grades Instructional Resources...73 References...75 i

6 This page intentionally left blank

7 Overview and Administration Guidelines 1

8 This page intentionally left blank

9 Florida Assessments for Instruction in Reading: Grades 3-12 Overview The assessment-for-learning system described in this manual is offered to schools as an aid for guiding literacy instruction for all students, but particularly for those who are struggling to meet grade level standards in late elementary, middle, and high school (grades 3-12). The assessments in this 3-12 system are meant to help improve learning by informing instruction; they are not part of accountability systems at the classroom, school, and district level. The most widely administered tasks in the system are computer-based for ease of administration and data management. These tasks are often referred to as the Web-based Assessment Module (WAM). Informal assessment procedures are also provided for use by individual teachers. In the current system, there are three computer based tasks. Broad Screen/Progress Monitoring Tool (BS/PMT) focused on grade-level skills in reading comprehension Administration of the BS/PMT three times a year satisfies the statutory requirement to monitor progress of all students who achieved Florida Comprehensive Assessment Test (FCAT) performance at level 1 or 2 in the previous year or who have been identified with a reading deficiency. This is a computer-adaptive measure of reading comprehension in which the passages and items mimic, as closely as possible, those that a student is likely to experience on the end-of-year FCAT. It provides four types of information to help plan instruction. 1. Current reading ability. A percentile rank score provides information about the student s current reading comprehension ability in comparison to a representative sample of students at the same grade level who took the task at the same time throughout Florida. A standard score will also be provided, which is standardized to have a mean of 100 and a standard deviation of 15. Teachers will also be provided with a developmental ability score, which is an Item Response Theory (IRT)-based score that describes the student s absolute ability on this task with a range from third to tenth grade ability. As a student increases in reading comprehension both within the year and across years, this score should show improvement. 2. FCAT Success Probability (FSP). This score indicates the probability that the student will be able to score at level 3 or above on FCAT Reading by the time the FCAT is taken in the spring. 3. Lexile range. This range is based on The Lexile Framework for Reading, and it indicates the level of text, in Lexile terms, that the student should be able to read with 75% comprehension. Cluster area scores. 4. The student s performance on each of the cluster areas assessed by the FCAT will be categorized as high, medium, or low. If there were not enough items given within a cluster area, the student score will be NA. 3

10 For students without previous FCAT scores, this task can be used at the beginning of the year to provide a reliable estimate of current reading ability relative to grade level standards in Florida. It could also be used to provide an updated estimate (in the context of standards for the student s current grade level) for students whose performance on the previous year s FCAT was marginal or below standards. The task is engineered to be taken in the fall and in the winter in order to gauge progress of individual students toward being able to meet grade-level standards in the spring. The most common use of the task should be to identify students in need of extra support at the beginning of the year and to evaluate progress on grade-level standards resulting from those supports during the year. For students who have an FCAT score from the previous year, information about the score on the FCAT will be combined with the score on this computer adaptive task to estimate the probability of success on the FCAT in the spring of the current academic year. Targeted Diagnostic Inventory (TDI) This diagnostic battery consists of two tasks whose purpose is to differentiate between students needing intensive daily intervention focused on a broad range of reading skills vs. students who can profit from less intensive intervention delivered by specially trained teachers within content areas. The focus of this latter type of intervention is on the higher-level comprehension and vocabulary skills required for success on the FCAT at the current grade level, rather than on basic reading skills like word analysis, reading fluency, and basic reading comprehension. The first task is called a Maze task, and it requires students to read two grade-level passages on the computer and respond to embedded cloze items within the text. The student chooses which of the three presented words is most consistent with the meaning of the sentence that is currently being read. It is a timed measure (3 minutes per passage) so it provides an estimate of silent reading fluency as well as low-level, or gist comprehension. This measure corresponds closely to the kind of Maze task recommended in a recent review of curriculum based measures by Wayman and colleagues (2007) as being appropriate for monitoring progress in basic reading skills in middle and high school. Two types of scores will be provided from this task: 1. Adjusted Maze Score. This is the average number of Maze items completed within three minutes in the two passages. It is corrected for guessing, and it is adjusted based on the difficulty of individual passages so that a more precise estimate of improvement from one task administration to another can be obtained. 2. Percentile rank and standard score. These scores compare the performance of the student to a representative sample of other students in the same grade in Florida who took the task at approximately the same time. The standard score is standardized to have a mean of 100 and a standard deviation of 15. It is important to note that the adjusted Maze score for this task is not the same as a words-correct- per minute (WCPM) score on an oral reading fluency probe. The adjusted Maze score reports the average number of Maze items completed correctly, and there is approximately one Maze item for every seven words in the passage. The Maze score will increase as the student becomes more fluent in reading the passage, as long as reading comprehension stays the same or increases. 4 The Word Analysis task within the Targeted Diagnostic Inventory is conceptualized as assessing the student s phonological, orthographic, and morphological knowledge of words in print. This task is computer-adaptive and will produce two kinds of scores: 2009 State of Florida, Department of Education. All Rights Reserved.

11 1. Developmental ability score. This IRT-based score identifies the student s absolute ability on this task with a range from third to tenth grade ability. As a student increases in knowledge of orthographic rules and patterns both within the year and across years, this score should show improvement. The developmental ability score associated with the Word Analysis task is reported as the Word Analysis Ability Score (WAAS). 2. Percentile rank and standard score. These scores compare the performance of the student to a representative sample of other students in the same grade level in Florida who took the task at approximately the same time. The standard score is standardized to have a mean of 100 and a standard deviation of 15. With further analysis of student outcomes on these tasks, at both the district and state level, it will be possible to provide more complete guidance about using these tasks to discriminate between at risk students who are struggling with basic reading skills (low-level comprehension, reading accuracy, and reading fluency) vs. those whose projected difficulties on the FCAT are primarily the result of difficulties with the complex reasoning and vocabulary demands of the FCAT at their current grade level. Detailed information will be provided about performance patterns on the Word Analysis task that may have direct implications for instruction on word reading accuracy. Ongoing Progress Monitoring (OPM) If intervention teachers working with struggling readers to improve basic reading processes would like to monitor the progress of their students more frequently than at the three major assessment points (Fall, Winter, Spring), the use of the Reading Comprehension or Maze tasks are recommended for this purpose in grades For grades 3 through 5, teachers can choose to use the Reading Comprehension Task, Oral Reading Fluency probes, or the Maze task. Off-grade level progress monitoring may be assigned because, for some students, reading passages at their grade level may be too frustrating, and may not be sensitive to growth in fluency or reading efficiency at lower levels of reading ability. These tasks can be administered at one month intervals. If teachers want to provide ongoing progress monitoring for students reading close to grade level, it is recommended to use the Broad Screen tool (Reading Comprehension Task) for this purpose. An adapted version of this tool can be administered at monthly intervals between the three major assessment points. Informal Diagnostic/Progress Assessment for Grades 3-12 Tool Kit These tasks and assessment procedures are for use primarily by intervention teachers, but in some cases, they can also be used by content area teachers to aid in day-to-day decision making in the classroom. These tasks and assessment procedures are paper/pencil and scores are not entered into the PMRN. Current tasks in this Tool Kit include: 1. Ongoing Progress Monitoring Oral Reading Fluency Grades 1-5. These tasks consist of multiple passages that represent the tasks in the Targeted Diagnostic Inventory that can be administered between Fall (AP 1), Winter (AP 2), and Spring (AP 3) Assessment Periods to provide more frequent monitoring of student progress in oral reading fluency. OPM tasks will also include equated, short passages for assessing oral reading fluency in one minute. 2. Instructional level passages with Scaffolded Discussion Templates. For students scoring below 5

12 the 30th percentile on the Maze and/or Word Analysis tasks, reading intervention teachers may use a timed Word Placement List to place students into a set of instructional passages that a student will be able to read with at least 90% accuracy. In this way, decoding can be controlled so that comprehension can be the focus of instruction. The teacher may select one passage from the set to use with a student or group of students. Accompanying these passages are Scaffolded Discussion Templates that the teacher can use to scaffold students understanding of the passage. Additionally, these passages have been equated for difficulty and, therefore, may be used to monitor fluency. Equated fluency scores are available in a look-up table at 3. Lexiled text passages. For students scoring above the 30th percentile on the Maze task and Word Analysis task but who are still challenged by comprehending grade-level text, teachers may use their Lexile range obtained from the Reading Comprehension Screen to match them to Lexiled passages in the tool kit. Reading intervention or content area teachers may group students assigned to the same passage and ask them to read silently and discuss answers to the comprehension questions. These passages have not been precisely equated for difficulty within grade level, so they should not be used for Progress Monitoring of Oral Reading Fluency. 4. Academic word inventory. This includes lists of academic words that occur with increasing frequency at each grade level to provide the teacher with an estimate of the student s context-free word reading ability. It is to be used by intensive intervention teachers for students scoring low on the Maze task and/or the Word Analysis task below the 30 th percentile. 5. Phonics screening inventory. This includes items and procedures for assessing the specific knowledge required for decoding words phonemically. Letter-sound correspondences, blending ability, skills with complex syllables or different syllable types are examples of the types of skills assessed by this inventory. It is meant to be used by intensive intervention teachers with students who have scored very poorly on the Word Analysis task in the Targeted Diagnostic Inventory. Data Management and Reporting Data from the formal, computer-based assessments will automatically enter the PMRN, which is Florida s web-based data management system that provides reports of student progress at the individual student, class, teacher, school, district, and state level. This information is immediately and constantly available to assist instructional personnel at all levels in making decisions about individual students, classrooms that need support, or school and district level progress in literacy instruction State of Florida, Department of Education. All Rights Reserved.

13 Florida Assessments for Instruction in Reading Grades 3-12 Conceptual Model Grades 3-12 Broad Screen/Progress Monitoring Tool Reading Comprehension Task Diagnostic Tool Kit (As Needed) If necessary Targeted Diagnostic Inventory Maze & Word Analysis tasks Ongoing Progress Monitoring (As Needed) Note: The Broad Screen/Progress Monitoring Tool, Targeted Diagnostic Inventory, and some Ongoing Progress Monitoring tasks require computer administration. Administration of the Broad Screen/ Progress Monitoring Tool three times a year satisfies the statutory requirement to monitor the progress of all students who achieved FCAT performance at level 1 or 2 in the previous year or who have been identified with a reading deficiency. 7

14 Web-based Assessment Module (WAM) Task Workflow Assessment 1 Fall Start Assessment 1 Student takes RC Screen FSP < 85%? No Take optional Maze? Yes Yes No Student takes Maze Student takes Maze Yes Take optional Word Analysis? Student takes Word Analysis No Stop State of Florida, Department of Education. All Rights Reserved.

15 Web-based Assessment Module (WAM) Task Workflow Assessment 2 Winter Start Assessment 2 Student takes RC Screen FSP < 85%? No FSP < 85% in Assessment 1? No Take optional Maze? Yes Yes Yes No Student takes Maze Student takes Maze Student takes Word Analysis Take optional Word Analysis? No Yes Student takes Word Analysis Stop 9

16 Web-based Assessment Module (WAM) Task Workflow Assessment 3 Spring Start Assessment 3 Student takes RC Screen Student takes Maze FSP < 85%? No Take optional Word Analysis? Yes Yes Student takes Word Analysis No Stop State of Florida, Department of Education. All Rights Reserved.

17 Technical Requirements Software Requirements are: Internet Explorer 6.0 or later (Windows platform only) Safari version 2.0 or later (preferred for Mac users) Mozilla Firefox version 1.5 or later (Mac or PC) Flash Player 9.0 or later JavaScript and Cookies enabled Hardware requirements are: PC Minimum Intel Pentium II 450MHz processor, 128MB of RAM, Screen Resolution of 1024 x 600 Recommended Intel Pentium III 1GHz or faster processor, 512MB of RAM, Screen Resolution of 1024 x 768 or higher Apple Intel Core Duo 1.33GHz processor PowerPC G3 500MHz processor 128MB of RAM, Screen Resolution of 1024 x 600 Intel Core Duo 1.83GHz or faster processor; PowerPC G4 1GHz or faster processor 512MB of RAM, Screen Resolution of 1024 x 768 or higher Students must have a set of headphones and a well functioning mouse to take the WAM. 11

18 This page intentionally left blank

19 Assessment Timeline Assessment Period 1 Fall Rostering Data Collection Period 1 Extended Window 1 No BS/PMT or TDI available Administer BS/PMT and TDI with scores assigned to student. Scores are aggregated up to the State Level. Students that enter the school during this time can still be assessed. Scores are not aggregated. Assessment Period 2 Winter Rostering Data Collection Period 2 Extended Window 2 No BS/PMT or TDI available Administer BS/PMT and TDI with scores assigned to student. Scores are aggregated up to the State Level. Students that enter the school during this time can still be assessed. Scores are not aggregated. Assessment Period 3 Spring Rostering Data Collection Period 3 Extended Window 3 No BS/PMT or TDI available Administer BS/PMT and TDI with scores assigned to student. Scores are aggregated up to the State Level. Students that enter the school during this time can still be assessed. Scores are not aggregated. 13

20 This page intentionally left blank

21 Roster Confirmation Prior to Testing The PMRN Web-based Assessment Module for grades 3-12 (3-12 WAM) is accessible by the educators and students at registered schools, and students must have a valid School Key to Sign In. The Key is a unique sequence of numeric and alphabetic values. It is not case-sensitive and is not associated with the student session. Each school has a unique Key which will change on a daily basis. The Key will only be valid for the school day between 5:00 am and 5:30 pm EST. Prior to obtaining the School Key, the School Level User must create a class for the reading teacher and grant him or her access to the PMRN. An will be sent to the reading teacher containing his or her User Name and Password for accessing the PMRN. The teacher will need to Sign In to the PMRN and change his or her Password. The User Name and new Password will be used to access the PMRN and the 3-12 WAM. The reading teacher should print class lists as they are in the PMRN. Although a class may be made up of multiple grades, the PMRN separates students by grades. Please also remember: 1. During an assessment period, a student can only be assigned to one reading teacher. 2. Imports of district data files assign students to teachers. For a student to be assigned to a class period, a teacher must make a request to the PMRN School Level 1, 2, or 3 user. After printing the class lists, reading teachers should check them for accuracy. Cross out the name of any student not in the class and add the full name and date of birth of students who should be on the list. The School Level 1, 2, or 3 user will then be able to make changes to the rosters. 15

22 Roster Confirmation Procedure To obtain the School Key, the WAM Manager must: Access the PMRN 3-12 WAM Home Page located at Notice the s after http. This indicates that the site is secure for the transmission of sensitive data. At the WAM Home Page, click on the For WAM Managers Only link, which will display the WAM Manager Sign In page. Enter your PMRN User Name and Password, and click Sign In. If you ve forgotten your Password, please click the Forgot Your Password? link and follow the instructions displayed State of Florida, Department of Education. All Rights Reserved.

23 If assigned to only one school and calendar, you will be taken to the Key Retrieval page. You are taken to the School or Calendar Selection page if you are assigned to more than one school or calendar. On the School or Calendar Selection page, click the radio button that corresponds to the correct school and calendar, and then press the Get Key button. The Key Retrieval Page is displayed, along with your School Key. The School Key may be copied. Press the Sign Out button. Once you have signed out, click here to Sign In again. 17

24 Teachers should use the Key to view the list of students at the school and to verify that the students to be assessed can access the tasks. To view the list of students: Access the 3-12 WAM located at Using your PMRN User Name and Password, click on the For WAM Managers Only link and Sign In. Click the Go to Student Page button to go to the WAM Home Page, where you will see the list of students. Confirm that all students scheduled to use the WAM appear on the list. For complete accuracy, please refer to the class lists of students that you printed earlier from the PMRN. Students Sign In to the 3-12 WAM by entering the School Key, which should be provided to them by their test proctor State of Florida, Department of Education. All Rights Reserved.

25 Web-based Assessment Module (WAM) Manager Functions The following PMRN User Access Levels can Sign In to the 3-12 Web Assessment Module (WAM): School Level 1 School Level 2 School Level 3 School Level 4 Reading Teacher Resource Teacher Assessment Team Member Sign In Access the 3-12 Web Assessment Module located at Notice the s after http. This indicates that the site is secure for the transmission of sensitive data. Enter your User Name and Password, and then click Sign in. Your PMRN Password must be reset in the PMRN before Signing In to the WAM. If you ve forgotten your User Name and Password, please click the Forgot Your Password? link and follow the instructions displayed. 19

26 Selection of School or Calendar If you are assigned to more than one school or calendar, you will be taken to the School or Calendar Selection page. On the School or Calendar Selection page, click the radio button that corresponds to the correct school and calendar, and click the Get Key button. Clicking the button will take you to the Key Retrieval page. Retrieval of School Key Your School Key will be displayed on the Key Retrieval page. The School Key may be copied. For reference, you may wish to write the key on a surface board near the computers the students are using. The students will need to enter the Key in order to Sign In. Click the Sign Out button to return to the WAM Manager Sign In page. 20 Click the Go to Student Page button to go to the WAM Home Page State of Florida, Department of Education. All Rights Reserved.

27 For rules concerning the School Key, please refer to the section in this manual, page, 16 Daily Sign In: Key Retrieval. Sound and Animation Check of Student Computers To verify that sound and animation function properly, administrators should check the computers the day before their students are assessed. To do this, please Sign In using the School Key, which will take you to the WAM Home Page. You may also visit the WAM Home Page through the link to conduct the Test Sound and Animation check at any time. Once at the WAM Home Page, click the Test Sound and Animation link to access the Sound and Animation Check page. The WAM will then determine whether or not a correct version of Flash is installed on the computer. If a correct version of Flash is installed, an image of a drum will appear on the page and you will hear the sound of a beating drum. The text Can you hear the drum? will be presented on the screen. If you can see and hear the drum, click the Yes button. If you are still having a problem viewing and hearing the drum, click the No button. 21

28 If you click the Yes button, you will be returned to the WAM Home Page. If you click the No button, the Try Again Speakers page will be displayed. You will see the following message: Please raise your hand for help. Your headphones, speakers, or computer may not be set-up properly. Click the Try Again button, and the Sound and Animation page will reload and attempt to verify that the correct version of Flash has been installed. The Try Again Flash page will now appear. You will see this message: Please raise your hand for help. You need Flash Player version 9.0 or higher on this computer State of Florida, Department of Education. All Rights Reserved.

29 To install the correct version of the Flash Player, click the Install Adobe Flash Player link. This link will navigate you to Adobe s web page for downloading the correct Flash Player software. Once you have downloaded the correct Flash Player, click the Try Again button. The Sound and Animation Check page will re-load and the WAM will verify that the Flash Player has been correctly installed. Confirmation of Students Check the WAM Home Page to confirm that all students scheduled to use the WAM appear on the student list that is displayed. Please check to make sure that each student s Last Name, First Name, and Grade Level is correct. Any updates to student rosters can be made in the PMRN. 23

30 This page intentionally left blank

31 Web-based Assessment Module (WAM) Student Functions Entry and Verification of School Key Students will need to access the 3-12 Web Assessment Module located at Notice the s after http. This indicates that the site is secure for the transmission of sensitive data. Students will use the School Key given the day they Sign In to the 3-12 WAM. For reference, please write the School Key on a surface board near the computers the students are using. Students will then enter the School Key in the space provided and click the Sign In button. Students have three attempts to Sign In to the WAM. After the third failed attempt, the student will have to close the browser window and re-open the WAM Home Page to Sign In. Sound and Animation Check Once the student has Signed In successfully, he or she will be taken to the Sound and Animation Check page. The WAM Manager can perform the Sound and Animation Check process on all computers before the students Sign In to the WAM. Please refer to the section, WAM Manager Functions, for instructions. Using the Sound and Animation Check page, the WAM will determine whether a correct version of Flash is installed on the computer. If a correct version of Flash is installed, an image of a drum will appear on the page, and the students will hear the sound of a beating drum. 25

32 The text Can you hear the drum? will appear on the screen. If the student can see and hear the drum, he or she should click the Yes button. If the student still has a problem viewing and hearing the drum, he or she should click the No button. If the student clicks the Yes button, he or she will be returned to the WAM Home Page. If the student clicks the No button, then the Try Again Speakers page will be displayed. A message will appear on the screen to instruct the student to Please raise your hand for help. Your headphones, speakers, or computer may not be set up properly. The WAM Manager will then Click the Try Again button, and the Sound and Animation page will reload and attempt to verify that the correct version of Flash has been installed. 26 The Try Again Flash page will now appear. A text message will read, Please raise your hand for help. You need Flash Player version 9.0 or higher on this computer State of Florida, Department of Education. All Rights Reserved.

33 To install the correct version of the Flash Player, the WAM Manager should click the Install Adobe Flash Player link. This link will navigate to Adobe s web page for downloading the correct Flash Player software. Once the WAM Manager has downloaded the correct Flash Player, he or she should click the Try Again button. The Sound and Animation Check page will re-load and the WAM will verify that the Flash Player has been correctly installed. 27

34 Name Selection and Confirmation The Student Selection page contains a list of students with their corresponding grade levels. Each student s name is a hyperlink. The student will locate his or her name in the list and click the hyperlink. A Student Confirmation page will now be displayed with a message stating, Are you (Student Name)? If the student s name is correct, he or she should click the Yes button, which will take him or her to the Student Birth Date page. If the student s name is not correct, he or she should click the No button, which will return him or her to the Student Selection page State of Florida, Department of Education. All Rights Reserved.

35 Entry of Birth Date and Verification of Student Identity On the Student Birth Date Selection page, the student should enter his or her date of birth by choosing the drop-down boxes corresponding to the correct Month, Day, and Year. After entering the birth date, the student should click the Start button. If the birth date is entered incorrectly, the student will see the following message: The date you have selected does not match our records. Please raise your hand for assistance. If the birth date is correct, the appropriate task for the student will be displayed. Administration of Appropriate Tasks The student will follow the instructions for the WAM tasks he or she is to complete. If any TDI Task is required for the student, the student must Sign Out and Sign In again in order to proceed to the Task. Tasks are assigned based on the WAM task Workflow, which is included in this manual on page If no other tasks are required, the teacher may direct the students to the optional MAZE or Word Analysis tasks. For a detailed description of the WAM tasks, please refer to the sections explaining the administration of the Reading Comprehension, Maze, and Word Analysis tasks on pages 33 and

36 This page intentionally left blank

37 Implementation Tips for Schools Scheduling (If administering in labs) Consider blocking out the lab schedules for all three assessment periods at the beginning of the year. See the PMRN data collection schedule to determine time frames. Consider scheduling students into the lab for one or two sessions. The Broad Screen/Progress Monitoring Tool Reading Comprehension Task takes minutes, depending on student performance. The Targeted Diagnostic Inventory (TDI) has two parts: The Maze task takes 10 minutes and the Word Analysis task takes minutes. Have students bring additional work to complete if they finish early. Administration of the Assessments Consider creating a shortcut to the assessment website for ease of administration. To create a shortcut for all types of platforms and browsers: 1) Go to 2) Click on the icon in the address bar, next to the URL address; 3) Drag it out to the desktop. Students will then be able to click on the shortcut for quick access to the assessments webpage. Make sure all new students have been rostered into the PMRN prior to taking the assessment. Otherwise, their name will not appear in the list of students. Sample Task Introduction Lab Session 1 ( Reading Comprehension): Today you will take some reading tasks on the computer. We ask that you help by taking this seriously and doing your best. You will begin by completing the Reading Comprehension Task. If you need help, raise your hand. After you Sign In, you will hear instructions through the set of headphones at your computer, and then you will complete the task. When you complete this section, raise your hand. Lab Session 2 ( Maze and Word Analysis). You will Sign In again to complete two quick tasks: a reading task called Maze and a spelling task called Word Analysis. Again, please take this seriously and do your best. If you need help, raise your hand. After you Sign In, you will hear instructions through the set of headphones at your computer, and then you will complete the task. Remember that the Maze task is timed, so choose the correct words quickly and accurately. For Technical or Support Questions PMRN Help Desk: (850) or toll free at (866) helpdesk@fcrr.org 31

38 This page intentionally left blank

39 Broad Screen/Progress Monitoring Tool (BS/PMT) 33

40 This page intentionally left blank

41 The Broad Screen / Progress Monitoring Tool (Reading Comprehension) Description and Purpose The Broad Screen/Progress Monitoring Tool consists of the Reading Comprehension Task. This task is designed to provide a valid, reliable, and efficient means for identifying students who are most likely to be able to meet grade level standards in reading at the end of the year. Performance on this task is predictive of success on the FCAT. The likelihood of this success is reported as a probability of success score (FCAT Success Probability, FSP), which falls on a Success Zone continuum. The Success Zone categories listed below differentiate student readers: Green Success Zone: The student has an 85% or greater probability of scoring at or above Level 3 on the current year s FCAT. Yellow Success Zone: The student has a 16-84% probability of scoring at or above Level 3 on the current year s FCAT. Red Success Zone: The student has a 1-15% probability of scoring at or above Level 3 on the current year s FCAT. The design of the Broad Screen/Progress Monitoring Tool Reading Comprehension Task intentionally mirrors the FCAT in both passage style and type of reading comprehension questions. Original passages are composed of either narrative or informational text. In this untimed task, the student is asked to read a passage and answer 7-9 corresponding comprehension questions. Based upon performance, the student will see an easier or more difficult passage until a reliable estimate of reading skill is reached (between 1-3 passages). 35

42 The Broad Screen / Progress Monitoring Tool Reading Comprehension Task Administration The Broad/Screen Progress Monitoring tool (BS/PMT) is an adaptive Reading Comprehension Task administered as an initial screen to students in grades The data from this task assists teachers in predicting the probablility of the student being successful on the reading portion of the Florida Comprehensive assessment Test (FCAT). The Web-based Assessment Module (WAM) will administer this task to all students enrolled in the PMRN at regular intervals three times per year. (Assessment periods 1, 2 and 3). The student will Sign In to the 3-12 WAM using the School Key that has been provided by the WAM Manager. The page will display the name of the task. When ready to begin, the student will click the Yes button. 36 The student will then encounter the Introduction and Practice Passage page. Audio tracks and accompanied text will guide the Student through the instructional pages to familiarize him or her with 2009 State of Florida, Department of Education. All Rights Reserved.

43 the functionality of the Reading Comprehension Task. When the students are ready to continue to the Reading Comprehension Task, they will click the Next button. The first passage displayed to the student is chosen based on the student s grade level and on information about the student s previous performance in reading. The student s previous year FCAT score as well as the FCAT Success Probability (FSP) from the most recent Reading Comprehension screen previously administered are used as indicators of the student s previous reading performance. The number and difficulty of passages are determined by the student s performance on the first passage. If there is no previous FCAT data for a student, the mean FCAT score for students in the same grade level at their school is used to determine the first RC passage displayed. A WAM Task Timeout Confirmation page with the message Your Session has timed out. Please Sign In to continue will appear if there is no computer activity for thirty minutes. Please click the Sign In link, and you will be directed to the WAM Home Page. The student will be administered no less than one passage and no more than three passages. At the end of each passage, the student must answer a series of questions by clicking the Answer Questions button. The questions relate to the passages and are both implicit and explicit. Not all students will encounter the same passages due to the adaptive nature of the Task. Once he or she is finished answering the questions, the student should click the Done button. 37

44 After each passage, the student will be taken to the Scoreboard page, which displays the number of questions attempted and the number of questions answered correctly. If the student has an FSP greater than.84 (>.84), the required testing for the student is complete. If the student is required to continue, he or she should click the Next button to continue to the next passage. If the student has successfully completed his or her passages, the student should click the Done button to advance to the Stop page. The Stop page congratulates the student for finishing the task and asks the student whether he or she would like to sign out or continue to the Maze task. There are now three scenarios for the student, which include: The student s tasks are complete. The teacher will assign optional tasks and the student will take them. The student will be required to take the Maze task during this session or another session State of Florida, Department of Education. All Rights Reserved.

45 Target Diagnostic Inventory (TDI) 39

46 This page intentionally left blank

47 The Targeted Diagnostic Inventory (Maze and Word Analysis) Description and Purpose Two tasks comprise the Targeted Diagnostic Inventory (TDI), the Maze and the Word Analysis tasks. Both tasks are designed to give teachers greater insight into the reading ability of students who fall within the Yellow or Red success zones by obtaining a probability of FCAT success score of 84% or lower. The Maze task measures text reading efficiency and gist-level comprehension. Text reading efficiency is a combined measure of both reading accuracy and reading speed both of these factors are necessary for good text comprehension. (i.e., the more efficiently the student can identify words in text, the better the comprehension). Successful readers must not only accurately recognize and decode words, but they must do so with relative ease and fluency (Armbruster, Lehr, & Osborn, 2001). Word Analysis provides an assessment of a student s understanding of the relationships between sounds and letters as well as spelling conventions based on orthographic and morphological patterns (Snow, Griffin, & Burns, 2005). Knowing how to spell is one indicator of a student s ability to successfully recognize, or decode words. Broader and deeper skills in applying letter and sound relationships, as well as orthographic and morphological knowledge support fluent and accurate reading, and consequently, reading comprehension. Together, results from the Maze and Word Analysis tasks will allow teachers to make informed instructional decisions regarding how best to support struggling readers. The Maze task will assist teachers in determining whether students with a low probability of success on the FCAT have fundamental difficulties with text reading efficiency and low level comprehension. The Word Analysis task will provide additional information about the extent of word-level reading difficulties in these same students. The combined information from these tasks will help school leaders and teachers accurately identify students who may need comprehensive and intensive interventions focused on a broad range of fundamental reading skills vs. those with reasonable basic reading skills who need extra help meeting grade level standards for the more complex types of comprehension assessed by the FCAT at their grade level. 41

48 Maze Task Administration The Maze task is a timed text reading efficiency task that is administered to students in grades Students must take the Maze task if their FCAT Success Probability (FSP) is 84% or lower, or if they were required to take the Maze task during a prior assessment period. During Assessment Period 3 (AP3), all students in grades 5-12 who are rostered in the PMRN are required to complete the Maze task. The Maze task provides teachers with knowledge about a student s text reading efficiency. The WAM administers the Maze task at regular intervals three times per year (Assessment periods 1, 2, and 3) to the students who meet the above criteria for Assessment Periods 1 and 2 and all 5 th -12 th grade students in Assessment Period 3. The Maze task may also be administered to students who are not required to take it State of Florida, Department of Education. All Rights Reserved.

49 The student will first encounter the Introduction and Practice Passage page. Audio Tracks and accompanying text will guide the student through the instructional pages to familiarize him or her with the functionality of the Maze task. When the student is ready to continue to the Maze task, he or she will click the Next button. The passages displayed to the student are pre-determined based on the assessment period and the grade level of the student. The student will be administered two passages that contain missing words. The student must fill in each missing word by choosing from three choices provided in a drop-down box for each blank. When the student is finished with the passage, he or she will click the Next button to continue to the next passage. If the student does not finish the passage in the three-minute time limit, he or she will automatically be taken to the Scoreboard page, which will display his or her results. After completing the second passage, the student will click the Done button. If the student attempts to click the Next or Done button before 75% of the items have been completed, a pop-up message will read, Please do all the items. 43

50 After each Maze passage, the student will be taken to the Scoreboard page, which displays the number of items the student attempted and the number of items he or she answered correctly State of Florida, Department of Education. All Rights Reserved.

51 Word Analysis Task Administration The Word Analysis (WA) task is an adaptive task where students are asked to spell a series of five to thirty words. The student will first hear the word, then the word used in a sentence, and finally, the word again. The first five words are pre-determined based on grade level and the remaining words are chosen based on the student s performance. Students with an FSP of 84% or lower are required to take the WA task. They may continue directly from the Maze task to the WA task or come back at another time. The WA task is optional for all students with an FSP of 85% or higher. The WAM administers the Word Analysis task at regular intervals three times per year (Assessment Periods 1, 2, and 3) to the students with the requirements listed above. If the student is not completing the WA task directly after the Maze task, the student will have to Sign In to the 3-12 WAM using the School Key that was provided by the WAM Manager. Confirmation will be given to the student as to which task he or she is about to take. If ready, the student will click the Yes button. 45

52 The student will first encounter the Introduction and Practice page. Audio Tracks and accompanying text will guide the student through the instructional pages to familiarize him or her with the functionality of the WA task. When the student is ready to continue to the WA task, he or she will click the Next button. The student will click the Ear graphic button to hear each word and example sentence read through an audio file. The student may click the Ear graphic button multiple times, if needed, to listen to the example again. After the audio file has finished playing, the student may type the word in the spelling box located below the Ear graphic button. After typing the word, the student will click the Done button, which will take him or her to the next word. A WAM task Timeout Confirmation page will appear if there is no computer activity for five minutes. The student will see the message, Your Session has timed out. Please Sign In to continue. The student must click the Sign In link and will be directed to the WAM Home Page. Due to the interruption, the WA task will restart at the Introduction page. If the interruption occurred before the fifth word was answered, the next word in the sequence (i.e., item 1, 2, 3, 4, or 5) will be presented. If the interruption occurred after the fifth word was answered, the next word presented will be based upon the student s performance up to that point. Once the student has successfully completed the Word Analysis task, the required assessments are complete and the Sign Out page will be displayed. The student will click the Sign Out button and return to the WAM Home Page State of Florida, Department of Education. All Rights Reserved.

53 Ongoing Progress Monitoring (OPM) 47

54 This page intentionally left blank

55 Ongoing Progress Monitoring (OPM) Reading Comprehension (RC) Ongoing Progress Monitoring Reading Comprehension The computer adaptive Reading Comprehension Task (Broad Screen/Progress Monitoring tool) included in the FAIR system is a potentially powerful way to monitor the progress of students who are receiving interventions focused on the type of complex comprehension skills that are assessed on the FCAT. Since both the passages and questions were written to mimic as closely as possible those that appear on the FCAT for each grade level, it should provide a relatively accurate estimate of growth toward grade level standards in reading. The Reading Comprehension Task can be given at all three major assessment periods (Fall, Winter, Spring) during the year, and a slightly modified version can be given more frequently (once every 20 instructional days), if desired. When the Reading Comprehension Task is used for ongoing progress monitoring, each student will receive two passages (one informational passage and one literary passage) which are chosen based on the student s ability on the prior assessment period s Reading Comprehension Task. The Ongoing Progress Monitoring Reading Comprehension Task is available to all students who participate in the FAIR system. In using the Tool to monitor progress, it is important to understand the meaning of the various scores provided from the task, since they mean different things in relationship to student progress. For example, the Developmental Ability Score (or ability score) is the most direct measure of improvement in abilities assessed by the task. This score reflects the student s reading skill on a scale that ranges from roughly 3 rd -10 th grade ability. If a student s absolute level of reading competence improves, either within a year or across years, it should be reflected by an increase in this score. Although this score is sensitive to improvements in reading ability, it does not provide information about whether the growth has been sufficient to increase the student s probability of success on the FCAT. The computer adaptive Reading Comprehension Task also provides estimates of strength and weakness in the four cluster categories assessed by the FCAT. However, since fewer items are used to assess each cluster category than occur on the total test, the estimates of ability in these individual areas are less reliable than is the overall score based on responses across all items. For this reason, students scores are simply categorized as high, medium, or low in each of the cluster categories. Although this data will be relatively reliable when considered at a classroom level (i.e. if a high percentage of the class are all weak in one particular area), for individual students it must be interpreted with caution, unless a consistent pattern of weakness or strength emerges over several different administrations of the test. OPM Maze When a student scores below the 30 th percentile on the Maze Targeted Diagnostic Inventory (TDI) task, there is reason for concern regarding the student s text reading efficiency. For students who are receiving a reading intervention in which one of the instructional goals is to increase reading efficiency (increase fluency and accuracy with basic comprehension), or for those students who scored below the 30 th percentile on the Maze TDI task, it may be appropriate to monitor growth more frequently than at the three major assessment points. It is recommended that a teacher progress monitor a student no more frequently than every 20 instructional days. 49

56 Students below the 30 th percentile on the Maze task from the TDI, students will be monitored on the OPM Maze task using passages that are below the student s current grade level. The use of easier passages may reduce frustration when the student is taking the task, and it may also allow more sensitivity to growth in text reading efficiency when the student s reading level is substantially below grade level. Based on this rationale, students will be administered passages at the following levels. If Maze task from TDI Score > 30 th percentile, no ongoing progress monitoring is available. If Maze task from TDI Score = 21 st -30 th percentile, student will be monitored on grade level (in grades 11-12, 10 th grade passages will be used). If Maze task from TDI Score = 20 th percentile or below: In grades 3-5, students will be monitored on grade level (in 3 rd ) or one grade below the student s current grade level. In grades 6-8, students will be monitored two grade levels below the student s current grade level. In grades 9-12, students will be monitored three grade levels below the student s current grade level. OPM Oral Reading Fluency (ORF) Grades 3-5 If the student is in grades 3-5, and if the teacher would rather have the student complete Ongoing Progress Monitoring using Oral Reading Fluency (ORF), it may be appropriate to monitor progress using passages that are below the student s current grade level. As with the OPM Maze passages, the use of easier passages may reduce frustration when the student is taking the test, and it may also allow more sensitivity to growth in fluency when the student s reading level is substantially below grade level. Based on this rationale, students will be administered passages at the following levels: If Maze task from TDI Score > 40 th percentile, no ongoing progress monitoring is available. If Maze task from TDI Score = 31 st -40 th percentile, student will be monitored on grade level. If Maze task from TDI Score = 21 st -30 th percentile, student will be monitored one grade below current grade level. If Maze task from TDI Score = 20 th percentile or below, student will be monitored two grades below current grade level State of Florida, Department of Education. All Rights Reserved.