Florida Assessments for Instruction in Reading Third Grade - Twelfth Grade

Transcription

1 Florida Assessments for Instruction in Reading Third Grade - Twelfth Grade Administration Manual

2 This product is protected by the copyright laws of the United States. Materials may not be copied, reproduced, republished, uploaded, posted, transmitted, distributed, or modified without the written consent of the Florida Department of Education, Tallahassee, Florida.

3 Contact Information Florida Department of Education Just Read, Florida! 325 W. Gaines Street, Suite 1548 Tallahassee, FL Florida Center for Reading Research Florida State University 2010 Levy Avenue, Suite 100 Tallahassee, FL (850)

4 This page intentionally left blank

5 Table of Contents Overview and Administration Guidelines...1 Grades 3-12 Assessments Overview...3 Assessment System Conceptual Model...7 Technical Requirements...11 Assessment Timeline...13 Roster Confirmation Prior to Testing...15 Roster Confirmation Procedure...16 Web-based Assessment Module (WAM) Manager Functions...19 Web-based Assessment Module (WAM) Student Functions...25 Implementation Tips for Schools...31 Broad Screen/ Progress Monitoring Tool (BS/PMT)...33 Assessment Description and Purpose...35 Reading Comprehension Task Administration...36 Targeted Diagnostic Inventory (TDI)...39 Assessment Description and Purpose...41 Maze Task Administration...42 Word Analysis Task Administration...45 Ongoing Progress Monitoring (OPM)...47 OPM Reading Comprehension...49 OPM Maze...49 OPM Oral Reading Fluency...50 Informal Diagnostic/Progress Assessment Tool Kit...51 Diagnostic Tool Kit Description and Directions for Use...53 Instructional Implications...63 Using Assessment to Guide Instruction...64 Appendix...69 Accommodations, Grades Instructional Resources...73 References...75 i

7 Overview and Administration Guidelines 1

9 Florida Assessments for Instruction in Reading: Grades 3-12 Overview The assessment-for-learning system described in this manual is offered to schools as an aid for guiding literacy instruction for all students, but particularly for those who are struggling to meet grade level standards in late elementary, middle, and high school (grades 3-12). The assessments in this 3-12 system are meant to help improve learning by informing instruction; they are not part of accountability systems at the classroom, school, and district level. The most widely administered tasks in the system are computer-based for ease of administration and data management. These tasks are often referred to as the Web-based Assessment Module (WAM). Informal assessment procedures are also provided for use by individual teachers. In the current system, there are three computer based tasks. Broad Screen/Progress Monitoring Tool (BS/PMT) focused on grade-level skills in reading comprehension Administration of the BS/PMT three times a year satisfies the statutory requirement to monitor progress of all students who achieved Florida Comprehensive Assessment Test (FCAT) performance at level 1 or 2 in the previous year or who have been identified with a reading deficiency. This is a computer-adaptive measure of reading comprehension in which the passages and items mimic, as closely as possible, those that a student is likely to experience on the end-of-year FCAT. It provides four types of information to help plan instruction. 1. Current reading ability. A percentile rank score provides information about the student s current reading comprehension ability in comparison to a representative sample of students at the same grade level who took the task at the same time throughout Florida. A standard score will also be provided, which is standardized to have a mean of 100 and a standard deviation of 15. Teachers will also be provided with a developmental ability score, which is an Item Response Theory (IRT)-based score that describes the student s absolute ability on this task with a range from third to tenth grade ability. As a student increases in reading comprehension both within the year and across years, this score should show improvement. 2. FCAT Success Probability (FSP). This score indicates the probability that the student will be able to score at level 3 or above on FCAT Reading by the time the FCAT is taken in the spring. 3. Lexile range. This range is based on The Lexile Framework for Reading, and it indicates the level of text, in Lexile terms, that the student should be able to read with 75% comprehension. Cluster area scores. 4. The student s performance on each of the cluster areas assessed by the FCAT will be categorized as high, medium, or low. If there were not enough items given within a cluster area, the student score will be NA. 3

10 For students without previous FCAT scores, this task can be used at the beginning of the year to provide a reliable estimate of current reading ability relative to grade level standards in Florida. It could also be used to provide an updated estimate (in the context of standards for the student s current grade level) for students whose performance on the previous year s FCAT was marginal or below standards. The task is engineered to be taken in the fall and in the winter in order to gauge progress of individual students toward being able to meet grade-level standards in the spring. The most common use of the task should be to identify students in need of extra support at the beginning of the year and to evaluate progress on grade-level standards resulting from those supports during the year. For students who have an FCAT score from the previous year, information about the score on the FCAT will be combined with the score on this computer adaptive task to estimate the probability of success on the FCAT in the spring of the current academic year. Targeted Diagnostic Inventory (TDI) This diagnostic battery consists of two tasks whose purpose is to differentiate between students needing intensive daily intervention focused on a broad range of reading skills vs. students who can profit from less intensive intervention delivered by specially trained teachers within content areas. The focus of this latter type of intervention is on the higher-level comprehension and vocabulary skills required for success on the FCAT at the current grade level, rather than on basic reading skills like word analysis, reading fluency, and basic reading comprehension. The first task is called a Maze task, and it requires students to read two grade-level passages on the computer and respond to embedded cloze items within the text. The student chooses which of the three presented words is most consistent with the meaning of the sentence that is currently being read. It is a timed measure (3 minutes per passage) so it provides an estimate of silent reading fluency as well as low-level, or gist comprehension. This measure corresponds closely to the kind of Maze task recommended in a recent review of curriculum based measures by Wayman and colleagues (2007) as being appropriate for monitoring progress in basic reading skills in middle and high school. Two types of scores will be provided from this task: 1. Adjusted Maze Score. This is the average number of Maze items completed within three minutes in the two passages. It is corrected for guessing, and it is adjusted based on the difficulty of individual passages so that a more precise estimate of improvement from one task administration to another can be obtained. 2. Percentile rank and standard score. These scores compare the performance of the student to a representative sample of other students in the same grade in Florida who took the task at approximately the same time. The standard score is standardized to have a mean of 100 and a standard deviation of 15. It is important to note that the adjusted Maze score for this task is not the same as a words-correct- per minute (WCPM) score on an oral reading fluency probe. The adjusted Maze score reports the average number of Maze items completed correctly, and there is approximately one Maze item for every seven words in the passage. The Maze score will increase as the student becomes more fluent in reading the passage, as long as reading comprehension stays the same or increases. 4 The Word Analysis task within the Targeted Diagnostic Inventory is conceptualized as assessing the student s phonological, orthographic, and morphological knowledge of words in print. This task is computer-adaptive and will produce two kinds of scores: 2009 State of Florida, Department of Education. All Rights Reserved.

11 1. Developmental ability score. This IRT-based score identifies the student s absolute ability on this task with a range from third to tenth grade ability. As a student increases in knowledge of orthographic rules and patterns both within the year and across years, this score should show improvement. The developmental ability score associated with the Word Analysis task is reported as the Word Analysis Ability Score (WAAS). 2. Percentile rank and standard score. These scores compare the performance of the student to a representative sample of other students in the same grade level in Florida who took the task at approximately the same time. The standard score is standardized to have a mean of 100 and a standard deviation of 15. With further analysis of student outcomes on these tasks, at both the district and state level, it will be possible to provide more complete guidance about using these tasks to discriminate between at risk students who are struggling with basic reading skills (low-level comprehension, reading accuracy, and reading fluency) vs. those whose projected difficulties on the FCAT are primarily the result of difficulties with the complex reasoning and vocabulary demands of the FCAT at their current grade level. Detailed information will be provided about performance patterns on the Word Analysis task that may have direct implications for instruction on word reading accuracy. Ongoing Progress Monitoring (OPM) If intervention teachers working with struggling readers to improve basic reading processes would like to monitor the progress of their students more frequently than at the three major assessment points (Fall, Winter, Spring), the use of the Reading Comprehension or Maze tasks are recommended for this purpose in grades For grades 3 through 5, teachers can choose to use the Reading Comprehension Task, Oral Reading Fluency probes, or the Maze task. Off-grade level progress monitoring may be assigned because, for some students, reading passages at their grade level may be too frustrating, and may not be sensitive to growth in fluency or reading efficiency at lower levels of reading ability. These tasks can be administered at one month intervals. If teachers want to provide ongoing progress monitoring for students reading close to grade level, it is recommended to use the Broad Screen tool (Reading Comprehension Task) for this purpose. An adapted version of this tool can be administered at monthly intervals between the three major assessment points. Informal Diagnostic/Progress Assessment for Grades 3-12 Tool Kit These tasks and assessment procedures are for use primarily by intervention teachers, but in some cases, they can also be used by content area teachers to aid in day-to-day decision making in the classroom. These tasks and assessment procedures are paper/pencil and scores are not entered into the PMRN. Current tasks in this Tool Kit include: 1. Ongoing Progress Monitoring Oral Reading Fluency Grades 1-5. These tasks consist of multiple passages that represent the tasks in the Targeted Diagnostic Inventory that can be administered between Fall (AP 1), Winter (AP 2), and Spring (AP 3) Assessment Periods to provide more frequent monitoring of student progress in oral reading fluency. OPM tasks will also include equated, short passages for assessing oral reading fluency in one minute. 2. Instructional level passages with Scaffolded Discussion Templates. For students scoring below 5

12 the 30th percentile on the Maze and/or Word Analysis tasks, reading intervention teachers may use a timed Word Placement List to place students into a set of instructional passages that a student will be able to read with at least 90% accuracy. In this way, decoding can be controlled so that comprehension can be the focus of instruction. The teacher may select one passage from the set to use with a student or group of students. Accompanying these passages are Scaffolded Discussion Templates that the teacher can use to scaffold students understanding of the passage. Additionally, these passages have been equated for difficulty and, therefore, may be used to monitor fluency. Equated fluency scores are available in a look-up table at 3. Lexiled text passages. For students scoring above the 30th percentile on the Maze task and Word Analysis task but who are still challenged by comprehending grade-level text, teachers may use their Lexile range obtained from the Reading Comprehension Screen to match them to Lexiled passages in the tool kit. Reading intervention or content area teachers may group students assigned to the same passage and ask them to read silently and discuss answers to the comprehension questions. These passages have not been precisely equated for difficulty within grade level, so they should not be used for Progress Monitoring of Oral Reading Fluency. 4. Academic word inventory. This includes lists of academic words that occur with increasing frequency at each grade level to provide the teacher with an estimate of the student s context-free word reading ability. It is to be used by intensive intervention teachers for students scoring low on the Maze task and/or the Word Analysis task below the 30 th percentile. 5. Phonics screening inventory. This includes items and procedures for assessing the specific knowledge required for decoding words phonemically. Letter-sound correspondences, blending ability, skills with complex syllables or different syllable types are examples of the types of skills assessed by this inventory. It is meant to be used by intensive intervention teachers with students who have scored very poorly on the Word Analysis task in the Targeted Diagnostic Inventory. Data Management and Reporting Data from the formal, computer-based assessments will automatically enter the PMRN, which is Florida s web-based data management system that provides reports of student progress at the individual student, class, teacher, school, district, and state level. This information is immediately and constantly available to assist instructional personnel at all levels in making decisions about individual students, classrooms that need support, or school and district level progress in literacy instruction State of Florida, Department of Education. All Rights Reserved.

13 Florida Assessments for Instruction in Reading Grades 3-12 Conceptual Model Grades 3-12 Broad Screen/Progress Monitoring Tool Reading Comprehension Task Diagnostic Tool Kit (As Needed) If necessary Targeted Diagnostic Inventory Maze & Word Analysis tasks Ongoing Progress Monitoring (As Needed) Note: The Broad Screen/Progress Monitoring Tool, Targeted Diagnostic Inventory, and some Ongoing Progress Monitoring tasks require computer administration. Administration of the Broad Screen/ Progress Monitoring Tool three times a year satisfies the statutory requirement to monitor the progress of all students who achieved FCAT performance at level 1 or 2 in the previous year or who have been identified with a reading deficiency. 7

14 Web-based Assessment Module (WAM) Task Workflow Assessment 1 Fall Start Assessment 1 Student takes RC Screen FSP < 85%? No Take optional Maze? Yes Yes No Student takes Maze Student takes Maze Yes Take optional Word Analysis? Student takes Word Analysis No Stop State of Florida, Department of Education. All Rights Reserved.

15 Web-based Assessment Module (WAM) Task Workflow Assessment 2 Winter Start Assessment 2 Student takes RC Screen FSP < 85%? No FSP < 85% in Assessment 1? No Take optional Maze? Yes Yes Yes No Student takes Maze Student takes Maze Student takes Word Analysis Take optional Word Analysis? No Yes Student takes Word Analysis Stop 9

16 Web-based Assessment Module (WAM) Task Workflow Assessment 3 Spring Start Assessment 3 Student takes RC Screen Student takes Maze FSP < 85%? No Take optional Word Analysis? Yes Yes Student takes Word Analysis No Stop State of Florida, Department of Education. All Rights Reserved.

17 Technical Requirements Software Requirements are: Internet Explorer 6.0 or later (Windows platform only) Safari version 2.0 or later (preferred for Mac users) Mozilla Firefox version 1.5 or later (Mac or PC) Flash Player 9.0 or later JavaScript and Cookies enabled Hardware requirements are: PC Minimum Intel Pentium II 450MHz processor, 128MB of RAM, Screen Resolution of 1024 x 600 Recommended Intel Pentium III 1GHz or faster processor, 512MB of RAM, Screen Resolution of 1024 x 768 or higher Apple Intel Core Duo 1.33GHz processor PowerPC G3 500MHz processor 128MB of RAM, Screen Resolution of 1024 x 600 Intel Core Duo 1.83GHz or faster processor; PowerPC G4 1GHz or faster processor 512MB of RAM, Screen Resolution of 1024 x 768 or higher Students must have a set of headphones and a well functioning mouse to take the WAM. 11

19 Assessment Timeline Assessment Period 1 Fall Rostering Data Collection Period 1 Extended Window 1 No BS/PMT or TDI available Administer BS/PMT and TDI with scores assigned to student. Scores are aggregated up to the State Level. Students that enter the school during this time can still be assessed. Scores are not aggregated. Assessment Period 2 Winter Rostering Data Collection Period 2 Extended Window 2 No BS/PMT or TDI available Administer BS/PMT and TDI with scores assigned to student. Scores are aggregated up to the State Level. Students that enter the school during this time can still be assessed. Scores are not aggregated. Assessment Period 3 Spring Rostering Data Collection Period 3 Extended Window 3 No BS/PMT or TDI available Administer BS/PMT and TDI with scores assigned to student. Scores are aggregated up to the State Level. Students that enter the school during this time can still be assessed. Scores are not aggregated. 13

21 Roster Confirmation Prior to Testing The PMRN Web-based Assessment Module for grades 3-12 (3-12 WAM) is accessible by the educators and students at registered schools, and students must have a valid School Key to Sign In. The Key is a unique sequence of numeric and alphabetic values. It is not case-sensitive and is not associated with the student session. Each school has a unique Key which will change on a daily basis. The Key will only be valid for the school day between 5:00 am and 5:30 pm EST. Prior to obtaining the School Key, the School Level User must create a class for the reading teacher and grant him or her access to the PMRN. An will be sent to the reading teacher containing his or her User Name and Password for accessing the PMRN. The teacher will need to Sign In to the PMRN and change his or her Password. The User Name and new Password will be used to access the PMRN and the 3-12 WAM. The reading teacher should print class lists as they are in the PMRN. Although a class may be made up of multiple grades, the PMRN separates students by grades. Please also remember: 1. During an assessment period, a student can only be assigned to one reading teacher. 2. Imports of district data files assign students to teachers. For a student to be assigned to a class period, a teacher must make a request to the PMRN School Level 1, 2, or 3 user. After printing the class lists, reading teachers should check them for accuracy. Cross out the name of any student not in the class and add the full name and date of birth of students who should be on the list. The School Level 1, 2, or 3 user will then be able to make changes to the rosters. 15

22 Roster Confirmation Procedure To obtain the School Key, the WAM Manager must: Access the PMRN 3-12 WAM Home Page located at Notice the s after http. This indicates that the site is secure for the transmission of sensitive data. At the WAM Home Page, click on the For WAM Managers Only link, which will display the WAM Manager Sign In page. Enter your PMRN User Name and Password, and click Sign In. If you ve forgotten your Password, please click the Forgot Your Password? link and follow the instructions displayed State of Florida, Department of Education. All Rights Reserved.

23 If assigned to only one school and calendar, you will be taken to the Key Retrieval page. You are taken to the School or Calendar Selection page if you are assigned to more than one school or calendar. On the School or Calendar Selection page, click the radio button that corresponds to the correct school and calendar, and then press the Get Key button. The Key Retrieval Page is displayed, along with your School Key. The School Key may be copied. Press the Sign Out button. Once you have signed out, click here to Sign In again. 17

24 Teachers should use the Key to view the list of students at the school and to verify that the students to be assessed can access the tasks. To view the list of students: Access the 3-12 WAM located at Using your PMRN User Name and Password, click on the For WAM Managers Only link and Sign In. Click the Go to Student Page button to go to the WAM Home Page, where you will see the list of students. Confirm that all students scheduled to use the WAM appear on the list. For complete accuracy, please refer to the class lists of students that you printed earlier from the PMRN. Students Sign In to the 3-12 WAM by entering the School Key, which should be provided to them by their test proctor State of Florida, Department of Education. All Rights Reserved.

25 Web-based Assessment Module (WAM) Manager Functions The following PMRN User Access Levels can Sign In to the 3-12 Web Assessment Module (WAM): School Level 1 School Level 2 School Level 3 School Level 4 Reading Teacher Resource Teacher Assessment Team Member Sign In Access the 3-12 Web Assessment Module located at Notice the s after http. This indicates that the site is secure for the transmission of sensitive data. Enter your User Name and Password, and then click Sign in. Your PMRN Password must be reset in the PMRN before Signing In to the WAM. If you ve forgotten your User Name and Password, please click the Forgot Your Password? link and follow the instructions displayed. 19

26 Selection of School or Calendar If you are assigned to more than one school or calendar, you will be taken to the School or Calendar Selection page. On the School or Calendar Selection page, click the radio button that corresponds to the correct school and calendar, and click the Get Key button. Clicking the button will take you to the Key Retrieval page. Retrieval of School Key Your School Key will be displayed on the Key Retrieval page. The School Key may be copied. For reference, you may wish to write the key on a surface board near the computers the students are using. The students will need to enter the Key in order to Sign In. Click the Sign Out button to return to the WAM Manager Sign In page. 20 Click the Go to Student Page button to go to the WAM Home Page State of Florida, Department of Education. All Rights Reserved.

27 For rules concerning the School Key, please refer to the section in this manual, page, 16 Daily Sign In: Key Retrieval. Sound and Animation Check of Student Computers To verify that sound and animation function properly, administrators should check the computers the day before their students are assessed. To do this, please Sign In using the School Key, which will take you to the WAM Home Page. You may also visit the WAM Home Page through the link to conduct the Test Sound and Animation check at any time. Once at the WAM Home Page, click the Test Sound and Animation link to access the Sound and Animation Check page. The WAM will then determine whether or not a correct version of Flash is installed on the computer. If a correct version of Flash is installed, an image of a drum will appear on the page and you will hear the sound of a beating drum. The text Can you hear the drum? will be presented on the screen. If you can see and hear the drum, click the Yes button. If you are still having a problem viewing and hearing the drum, click the No button. 21

28 If you click the Yes button, you will be returned to the WAM Home Page. If you click the No button, the Try Again Speakers page will be displayed. You will see the following message: Please raise your hand for help. Your headphones, speakers, or computer may not be set-up properly. Click the Try Again button, and the Sound and Animation page will reload and attempt to verify that the correct version of Flash has been installed. The Try Again Flash page will now appear. You will see this message: Please raise your hand for help. You need Flash Player version 9.0 or higher on this computer State of Florida, Department of Education. All Rights Reserved.

29 To install the correct version of the Flash Player, click the Install Adobe Flash Player link. This link will navigate you to Adobe s web page for downloading the correct Flash Player software. Once you have downloaded the correct Flash Player, click the Try Again button. The Sound and Animation Check page will re-load and the WAM will verify that the Flash Player has been correctly installed. Confirmation of Students Check the WAM Home Page to confirm that all students scheduled to use the WAM appear on the student list that is displayed. Please check to make sure that each student s Last Name, First Name, and Grade Level is correct. Any updates to student rosters can be made in the PMRN. 23

31 Web-based Assessment Module (WAM) Student Functions Entry and Verification of School Key Students will need to access the 3-12 Web Assessment Module located at Notice the s after http. This indicates that the site is secure for the transmission of sensitive data. Students will use the School Key given the day they Sign In to the 3-12 WAM. For reference, please write the School Key on a surface board near the computers the students are using. Students will then enter the School Key in the space provided and click the Sign In button. Students have three attempts to Sign In to the WAM. After the third failed attempt, the student will have to close the browser window and re-open the WAM Home Page to Sign In. Sound and Animation Check Once the student has Signed In successfully, he or she will be taken to the Sound and Animation Check page. The WAM Manager can perform the Sound and Animation Check process on all computers before the students Sign In to the WAM. Please refer to the section, WAM Manager Functions, for instructions. Using the Sound and Animation Check page, the WAM will determine whether a correct version of Flash is installed on the computer. If a correct version of Flash is installed, an image of a drum will appear on the page, and the students will hear the sound of a beating drum. 25

32 The text Can you hear the drum? will appear on the screen. If the student can see and hear the drum, he or she should click the Yes button. If the student still has a problem viewing and hearing the drum, he or she should click the No button. If the student clicks the Yes button, he or she will be returned to the WAM Home Page. If the student clicks the No button, then the Try Again Speakers page will be displayed. A message will appear on the screen to instruct the student to Please raise your hand for help. Your headphones, speakers, or computer may not be set up properly. The WAM Manager will then Click the Try Again button, and the Sound and Animation page will reload and attempt to verify that the correct version of Flash has been installed. 26 The Try Again Flash page will now appear. A text message will read, Please raise your hand for help. You need Flash Player version 9.0 or higher on this computer State of Florida, Department of Education. All Rights Reserved.

33 To install the correct version of the Flash Player, the WAM Manager should click the Install Adobe Flash Player link. This link will navigate to Adobe s web page for downloading the correct Flash Player software. Once the WAM Manager has downloaded the correct Flash Player, he or she should click the Try Again button. The Sound and Animation Check page will re-load and the WAM will verify that the Flash Player has been correctly installed. 27

34 Name Selection and Confirmation The Student Selection page contains a list of students with their corresponding grade levels. Each student s name is a hyperlink. The student will locate his or her name in the list and click the hyperlink. A Student Confirmation page will now be displayed with a message stating, Are you (Student Name)? If the student s name is correct, he or she should click the Yes button, which will take him or her to the Student Birth Date page. If the student s name is not correct, he or she should click the No button, which will return him or her to the Student Selection page State of Florida, Department of Education. All Rights Reserved.

35 Entry of Birth Date and Verification of Student Identity On the Student Birth Date Selection page, the student should enter his or her date of birth by choosing the drop-down boxes corresponding to the correct Month, Day, and Year. After entering the birth date, the student should click the Start button. If the birth date is entered incorrectly, the student will see the following message: The date you have selected does not match our records. Please raise your hand for assistance. If the birth date is correct, the appropriate task for the student will be displayed. Administration of Appropriate Tasks The student will follow the instructions for the WAM tasks he or she is to complete. If any TDI Task is required for the student, the student must Sign Out and Sign In again in order to proceed to the Task. Tasks are assigned based on the WAM task Workflow, which is included in this manual on page If no other tasks are required, the teacher may direct the students to the optional MAZE or Word Analysis tasks. For a detailed description of the WAM tasks, please refer to the sections explaining the administration of the Reading Comprehension, Maze, and Word Analysis tasks on pages 33 and

37 Implementation Tips for Schools Scheduling (If administering in labs) Consider blocking out the lab schedules for all three assessment periods at the beginning of the year. See the PMRN data collection schedule to determine time frames. Consider scheduling students into the lab for one or two sessions. The Broad Screen/Progress Monitoring Tool Reading Comprehension Task takes minutes, depending on student performance. The Targeted Diagnostic Inventory (TDI) has two parts: The Maze task takes 10 minutes and the Word Analysis task takes minutes. Have students bring additional work to complete if they finish early. Administration of the Assessments Consider creating a shortcut to the assessment website for ease of administration. To create a shortcut for all types of platforms and browsers: 1) Go to 2) Click on the icon in the address bar, next to the URL address; 3) Drag it out to the desktop. Students will then be able to click on the shortcut for quick access to the assessments webpage. Make sure all new students have been rostered into the PMRN prior to taking the assessment. Otherwise, their name will not appear in the list of students. Sample Task Introduction Lab Session 1 ( Reading Comprehension): Today you will take some reading tasks on the computer. We ask that you help by taking this seriously and doing your best. You will begin by completing the Reading Comprehension Task. If you need help, raise your hand. After you Sign In, you will hear instructions through the set of headphones at your computer, and then you will complete the task. When you complete this section, raise your hand. Lab Session 2 ( Maze and Word Analysis). You will Sign In again to complete two quick tasks: a reading task called Maze and a spelling task called Word Analysis. Again, please take this seriously and do your best. If you need help, raise your hand. After you Sign In, you will hear instructions through the set of headphones at your computer, and then you will complete the task. Remember that the Maze task is timed, so choose the correct words quickly and accurately. For Technical or Support Questions PMRN Help Desk: (850) or toll free at (866) [email protected] 31

39 Broad Screen/Progress Monitoring Tool (BS/PMT) 33

41 The Broad Screen / Progress Monitoring Tool (Reading Comprehension) Description and Purpose The Broad Screen/Progress Monitoring Tool consists of the Reading Comprehension Task. This task is designed to provide a valid, reliable, and efficient means for identifying students who are most likely to be able to meet grade level standards in reading at the end of the year. Performance on this task is predictive of success on the FCAT. The likelihood of this success is reported as a probability of success score (FCAT Success Probability, FSP), which falls on a Success Zone continuum. The Success Zone categories listed below differentiate student readers: Green Success Zone: The student has an 85% or greater probability of scoring at or above Level 3 on the current year s FCAT. Yellow Success Zone: The student has a 16-84% probability of scoring at or above Level 3 on the current year s FCAT. Red Success Zone: The student has a 1-15% probability of scoring at or above Level 3 on the current year s FCAT. The design of the Broad Screen/Progress Monitoring Tool Reading Comprehension Task intentionally mirrors the FCAT in both passage style and type of reading comprehension questions. Original passages are composed of either narrative or informational text. In this untimed task, the student is asked to read a passage and answer 7-9 corresponding comprehension questions. Based upon performance, the student will see an easier or more difficult passage until a reliable estimate of reading skill is reached (between 1-3 passages). 35

42 The Broad Screen / Progress Monitoring Tool Reading Comprehension Task Administration The Broad/Screen Progress Monitoring tool (BS/PMT) is an adaptive Reading Comprehension Task administered as an initial screen to students in grades The data from this task assists teachers in predicting the probablility of the student being successful on the reading portion of the Florida Comprehensive assessment Test (FCAT). The Web-based Assessment Module (WAM) will administer this task to all students enrolled in the PMRN at regular intervals three times per year. (Assessment periods 1, 2 and 3). The student will Sign In to the 3-12 WAM using the School Key that has been provided by the WAM Manager. The page will display the name of the task. When ready to begin, the student will click the Yes button. 36 The student will then encounter the Introduction and Practice Passage page. Audio tracks and accompanied text will guide the Student through the instructional pages to familiarize him or her with 2009 State of Florida, Department of Education. All Rights Reserved.

43 the functionality of the Reading Comprehension Task. When the students are ready to continue to the Reading Comprehension Task, they will click the Next button. The first passage displayed to the student is chosen based on the student s grade level and on information about the student s previous performance in reading. The student s previous year FCAT score as well as the FCAT Success Probability (FSP) from the most recent Reading Comprehension screen previously administered are used as indicators of the student s previous reading performance. The number and difficulty of passages are determined by the student s performance on the first passage. If there is no previous FCAT data for a student, the mean FCAT score for students in the same grade level at their school is used to determine the first RC passage displayed. A WAM Task Timeout Confirmation page with the message Your Session has timed out. Please Sign In to continue will appear if there is no computer activity for thirty minutes. Please click the Sign In link, and you will be directed to the WAM Home Page. The student will be administered no less than one passage and no more than three passages. At the end of each passage, the student must answer a series of questions by clicking the Answer Questions button. The questions relate to the passages and are both implicit and explicit. Not all students will encounter the same passages due to the adaptive nature of the Task. Once he or she is finished answering the questions, the student should click the Done button. 37

44 After each passage, the student will be taken to the Scoreboard page, which displays the number of questions attempted and the number of questions answered correctly. If the student has an FSP greater than.84 (>.84), the required testing for the student is complete. If the student is required to continue, he or she should click the Next button to continue to the next passage. If the student has successfully completed his or her passages, the student should click the Done button to advance to the Stop page. The Stop page congratulates the student for finishing the task and asks the student whether he or she would like to sign out or continue to the Maze task. There are now three scenarios for the student, which include: The student s tasks are complete. The teacher will assign optional tasks and the student will take them. The student will be required to take the Maze task during this session or another session State of Florida, Department of Education. All Rights Reserved.

45 Target Diagnostic Inventory (TDI) 39

47 The Targeted Diagnostic Inventory (Maze and Word Analysis) Description and Purpose Two tasks comprise the Targeted Diagnostic Inventory (TDI), the Maze and the Word Analysis tasks. Both tasks are designed to give teachers greater insight into the reading ability of students who fall within the Yellow or Red success zones by obtaining a probability of FCAT success score of 84% or lower. The Maze task measures text reading efficiency and gist-level comprehension. Text reading efficiency is a combined measure of both reading accuracy and reading speed both of these factors are necessary for good text comprehension. (i.e., the more efficiently the student can identify words in text, the better the comprehension). Successful readers must not only accurately recognize and decode words, but they must do so with relative ease and fluency (Armbruster, Lehr, & Osborn, 2001). Word Analysis provides an assessment of a student s understanding of the relationships between sounds and letters as well as spelling conventions based on orthographic and morphological patterns (Snow, Griffin, & Burns, 2005). Knowing how to spell is one indicator of a student s ability to successfully recognize, or decode words. Broader and deeper skills in applying letter and sound relationships, as well as orthographic and morphological knowledge support fluent and accurate reading, and consequently, reading comprehension. Together, results from the Maze and Word Analysis tasks will allow teachers to make informed instructional decisions regarding how best to support struggling readers. The Maze task will assist teachers in determining whether students with a low probability of success on the FCAT have fundamental difficulties with text reading efficiency and low level comprehension. The Word Analysis task will provide additional information about the extent of word-level reading difficulties in these same students. The combined information from these tasks will help school leaders and teachers accurately identify students who may need comprehensive and intensive interventions focused on a broad range of fundamental reading skills vs. those with reasonable basic reading skills who need extra help meeting grade level standards for the more complex types of comprehension assessed by the FCAT at their grade level. 41

48 Maze Task Administration The Maze task is a timed text reading efficiency task that is administered to students in grades Students must take the Maze task if their FCAT Success Probability (FSP) is 84% or lower, or if they were required to take the Maze task during a prior assessment period. During Assessment Period 3 (AP3), all students in grades 5-12 who are rostered in the PMRN are required to complete the Maze task. The Maze task provides teachers with knowledge about a student s text reading efficiency. The WAM administers the Maze task at regular intervals three times per year (Assessment periods 1, 2, and 3) to the students who meet the above criteria for Assessment Periods 1 and 2 and all 5 th -12 th grade students in Assessment Period 3. The Maze task may also be administered to students who are not required to take it State of Florida, Department of Education. All Rights Reserved.

49 The student will first encounter the Introduction and Practice Passage page. Audio Tracks and accompanying text will guide the student through the instructional pages to familiarize him or her with the functionality of the Maze task. When the student is ready to continue to the Maze task, he or she will click the Next button. The passages displayed to the student are pre-determined based on the assessment period and the grade level of the student. The student will be administered two passages that contain missing words. The student must fill in each missing word by choosing from three choices provided in a drop-down box for each blank. When the student is finished with the passage, he or she will click the Next button to continue to the next passage. If the student does not finish the passage in the three-minute time limit, he or she will automatically be taken to the Scoreboard page, which will display his or her results. After completing the second passage, the student will click the Done button. If the student attempts to click the Next or Done button before 75% of the items have been completed, a pop-up message will read, Please do all the items. 43

50 After each Maze passage, the student will be taken to the Scoreboard page, which displays the number of items the student attempted and the number of items he or she answered correctly State of Florida, Department of Education. All Rights Reserved.

51 Word Analysis Task Administration The Word Analysis (WA) task is an adaptive task where students are asked to spell a series of five to thirty words. The student will first hear the word, then the word used in a sentence, and finally, the word again. The first five words are pre-determined based on grade level and the remaining words are chosen based on the student s performance. Students with an FSP of 84% or lower are required to take the WA task. They may continue directly from the Maze task to the WA task or come back at another time. The WA task is optional for all students with an FSP of 85% or higher. The WAM administers the Word Analysis task at regular intervals three times per year (Assessment Periods 1, 2, and 3) to the students with the requirements listed above. If the student is not completing the WA task directly after the Maze task, the student will have to Sign In to the 3-12 WAM using the School Key that was provided by the WAM Manager. Confirmation will be given to the student as to which task he or she is about to take. If ready, the student will click the Yes button. 45

52 The student will first encounter the Introduction and Practice page. Audio Tracks and accompanying text will guide the student through the instructional pages to familiarize him or her with the functionality of the WA task. When the student is ready to continue to the WA task, he or she will click the Next button. The student will click the Ear graphic button to hear each word and example sentence read through an audio file. The student may click the Ear graphic button multiple times, if needed, to listen to the example again. After the audio file has finished playing, the student may type the word in the spelling box located below the Ear graphic button. After typing the word, the student will click the Done button, which will take him or her to the next word. A WAM task Timeout Confirmation page will appear if there is no computer activity for five minutes. The student will see the message, Your Session has timed out. Please Sign In to continue. The student must click the Sign In link and will be directed to the WAM Home Page. Due to the interruption, the WA task will restart at the Introduction page. If the interruption occurred before the fifth word was answered, the next word in the sequence (i.e., item 1, 2, 3, 4, or 5) will be presented. If the interruption occurred after the fifth word was answered, the next word presented will be based upon the student s performance up to that point. Once the student has successfully completed the Word Analysis task, the required assessments are complete and the Sign Out page will be displayed. The student will click the Sign Out button and return to the WAM Home Page State of Florida, Department of Education. All Rights Reserved.

53 Ongoing Progress Monitoring (OPM) 47

55 Ongoing Progress Monitoring (OPM) Reading Comprehension (RC) Ongoing Progress Monitoring Reading Comprehension The computer adaptive Reading Comprehension Task (Broad Screen/Progress Monitoring tool) included in the FAIR system is a potentially powerful way to monitor the progress of students who are receiving interventions focused on the type of complex comprehension skills that are assessed on the FCAT. Since both the passages and questions were written to mimic as closely as possible those that appear on the FCAT for each grade level, it should provide a relatively accurate estimate of growth toward grade level standards in reading. The Reading Comprehension Task can be given at all three major assessment periods (Fall, Winter, Spring) during the year, and a slightly modified version can be given more frequently (once every 20 instructional days), if desired. When the Reading Comprehension Task is used for ongoing progress monitoring, each student will receive two passages (one informational passage and one literary passage) which are chosen based on the student s ability on the prior assessment period s Reading Comprehension Task. The Ongoing Progress Monitoring Reading Comprehension Task is available to all students who participate in the FAIR system. In using the Tool to monitor progress, it is important to understand the meaning of the various scores provided from the task, since they mean different things in relationship to student progress. For example, the Developmental Ability Score (or ability score) is the most direct measure of improvement in abilities assessed by the task. This score reflects the student s reading skill on a scale that ranges from roughly 3 rd -10 th grade ability. If a student s absolute level of reading competence improves, either within a year or across years, it should be reflected by an increase in this score. Although this score is sensitive to improvements in reading ability, it does not provide information about whether the growth has been sufficient to increase the student s probability of success on the FCAT. The computer adaptive Reading Comprehension Task also provides estimates of strength and weakness in the four cluster categories assessed by the FCAT. However, since fewer items are used to assess each cluster category than occur on the total test, the estimates of ability in these individual areas are less reliable than is the overall score based on responses across all items. For this reason, students scores are simply categorized as high, medium, or low in each of the cluster categories. Although this data will be relatively reliable when considered at a classroom level (i.e. if a high percentage of the class are all weak in one particular area), for individual students it must be interpreted with caution, unless a consistent pattern of weakness or strength emerges over several different administrations of the test. OPM Maze When a student scores below the 30 th percentile on the Maze Targeted Diagnostic Inventory (TDI) task, there is reason for concern regarding the student s text reading efficiency. For students who are receiving a reading intervention in which one of the instructional goals is to increase reading efficiency (increase fluency and accuracy with basic comprehension), or for those students who scored below the 30 th percentile on the Maze TDI task, it may be appropriate to monitor growth more frequently than at the three major assessment points. It is recommended that a teacher progress monitor a student no more frequently than every 20 instructional days. 49

56 Students below the 30 th percentile on the Maze task from the TDI, students will be monitored on the OPM Maze task using passages that are below the student s current grade level. The use of easier passages may reduce frustration when the student is taking the task, and it may also allow more sensitivity to growth in text reading efficiency when the student s reading level is substantially below grade level. Based on this rationale, students will be administered passages at the following levels. If Maze task from TDI Score > 30 th percentile, no ongoing progress monitoring is available. If Maze task from TDI Score = 21 st -30 th percentile, student will be monitored on grade level (in grades 11-12, 10 th grade passages will be used). If Maze task from TDI Score = 20 th percentile or below: In grades 3-5, students will be monitored on grade level (in 3 rd ) or one grade below the student s current grade level. In grades 6-8, students will be monitored two grade levels below the student s current grade level. In grades 9-12, students will be monitored three grade levels below the student s current grade level. OPM Oral Reading Fluency (ORF) Grades 3-5 If the student is in grades 3-5, and if the teacher would rather have the student complete Ongoing Progress Monitoring using Oral Reading Fluency (ORF), it may be appropriate to monitor progress using passages that are below the student s current grade level. As with the OPM Maze passages, the use of easier passages may reduce frustration when the student is taking the test, and it may also allow more sensitivity to growth in fluency when the student s reading level is substantially below grade level. Based on this rationale, students will be administered passages at the following levels: If Maze task from TDI Score > 40 th percentile, no ongoing progress monitoring is available. If Maze task from TDI Score = 31 st -40 th percentile, student will be monitored on grade level. If Maze task from TDI Score = 21 st -30 th percentile, student will be monitored one grade below current grade level. If Maze task from TDI Score = 20 th percentile or below, student will be monitored two grades below current grade level State of Florida, Department of Education. All Rights Reserved.

57 3-12 Informal Diagnostic/Progress Assessment Tool Kit 51

59 3-12 Informal Diagnostic/Progress Assessment Tool Kit The diagnostic and instructional materials in the 3-12 Tool Kit are for use primarily by intervention teachers, but may also be used by content area teachers to aid in day-to-day decision making in the classroom. Only paper copies of these materials are provided and student responses are not entered into the PMRN. Current tasks in this Tool Kit include: 1. Ongoing Progress Monitoring Oral Reading Fluency Grades 1-5. These tasks consist of multiple passages that represent the tasks in the Targeted Diagnostic Inventory that can be administered between Fall (AP 1), Winter (AP 2), and Spring (AP 3) Assessment Periods to provide more frequent monitoring of student progress in oral reading fluency. OPM tasks will also include equated, short passages for assessing oral reading fluency in one minute. 2. Instructional level passages with Scaffolded Discussion Templates. For students scoring below the 30th percentile on the Maze and/or Word Analysis tasks, reading intervention teachers may use a timed Word Placement List to place students into a set of instructional passages that a student will be able to read with at least 90% accuracy. In this way, decoding can be controlled so that comprehension can be the focus of instruction. The teacher may select one passage from the set to use with a student or group of students. Accompanying these passages are Scaffolded Discussion Templates that the teacher can use to scaffold students understanding of the passage. Additionally, these passages have been equated for difficulty and, therefore, may be used to monitor fluency. Equated fluency scores are available in a look-up table at 3. Lexiled text passages. For students scoring above the 30th percentile on the Maze task and Word Analysis task but who are still challenged by comprehending grade-level text, teachers may use their Lexile range obtained from the Reading Comprehension Screen to match them to Lexiled passages in the tool kit. Reading intervention or content area teachers may group students assigned to the same passage and ask them to read silently and discuss answers to the comprehension questions. These passages have not been precisely equated for difficulty within grade level, so they should not be used for Progress Monitoring of Oral Reading Fluency. 4. Academic word inventory. This includes lists of academic words that occur with increasing frequency at each grade level to provide the teacher with an estimate of the student s contextfree word reading ability. It is to be used by intensive intervention teachers for students scoring low on the Maze task and/or the Word Analysis task below the 30 th percentile. P 5. honics screening inventory. This includes items and procedures for assessing the specific knowledge required for decoding words phonemically. Letter-sound correspondences, blending ability, skills with complex syllables or different syllable types are examples of the types of skills assessed by this inventory. It is meant to be used by intensive intervention teachers with students who have scored very poorly on the Word Analysis task in the Targeted Diagnostic Inventory. 53

60 Ongoing Progress Monitoring Oral Reading Fluency Ongoing progress monitoring (OPM) is a scientifically based assessment that is used periodically to assess student progress and the effectiveness of a chosen instructional practice. These measures are used when the teacher wants to monitor progress more frequently than the three Assessment Periods, which are considered progress monitoring. The time frame between OPM measures on individual tasks is set at 20 instructional days. This time period allows for potential growth to be measured. OPM is available to students who have a FCAT Success Probability (FSP) < 85% on the Broad Screen and districts and schools will determine their plan for selecting students to monitor. There are many instructional benefits for monitoring progress more frequently than three times a year. These include: Providing data driven instruction Offering documentation of student progress relating to Response to Intervention Providing up to date progress information to families OPM - Oral Reading Fluency (ORF) Probes Oral reading fluency passages have been provided for students in to monitor progress with student oral reading fluency. It is recommended that regardless of the Assessment Period, the oral reading fluency probes are used with students only after they have successfully completed the Reading Comprehension Broad Screen. These passages have been equated for difficulty to take into account the variability of difficulty in different passages (Francis, Santi, Barr, Fletcher, Varisco, & Foorman, 2008). This allows for comparison of scores among the passages. It is anticipated that teachers will use these passages to observe student use of strategies during reading and monitor oral reading fluency at students current reading level. A basic comprehension question is added at the end of the timing to get insight into a student s ability to monitor his/her comprehension. It also serves as a reminder to students that comprehension is the ultimate goal of reading State of Florida, Department of Education. All Rights Reserved.

61 Scaffolded Discussion Templates For students scoring below the 30th percentile on the Maze and/or Word Analysis tasks, reading intervention teachers may use a timed Word Placement List to place students into a set of instructional passages that a student will be able to read with at least 90% accuracy. In this way, decoding can be controlled so that comprehension can be the focus of instruction. The teacher may select one passage from the set to use with a student or group of students. Accompanying these passages are Scaffolded Discussion Templates that the teacher can use to scaffold students understanding of the passage. Additionally, these passages have been equated for difficulty and, therefore, may be used to monitor fluency. Equated fluency scores are available in a look-up table at Overview of Scaffolded Discussion Templates: Reading is the process of simultaneously extracting and constructing meaning through interaction and involvement with written language (RAND, 2002, p.11). These templates are designed to foster student engagement and involvement in the process of developing and interpreting meaning from written text. Each template utilizes two approaches to achieve this goal: y The first approach is based on the Question-Answer Relationships strategy (QAR) (Raphael, 1982; 1986). It is intended to assist with comprehension by teaching the student to use evidence from the text to answer questions and teaching the student to draw conclusions / make inferences based on textual information and one s own knowledge. y Purpose: y y y The second alternate thinking approach may be used when a student needs additional support. It addresses issues that may be unique to that specific question. The support is offered in the form of scaffolding, which explains and guides the student as he or she learns to analyze and determine various ways to answer a question. To provide scaffolding to students as they think about comprehension questions To provide ample practice with these approaches so students can learn to internalize and generalize them across contexts Possible uses: - Teaching and Assessment tool o Administer individually: read passage and use scaffolded discussion templates to analyze and answer comprehension questions o Administer to group: read passage and use scaffolded discussion templates to analyze and answer comprehension questions - Fluency measure and comprehension questions o Administer individually: obtain rate and accuracy measure and use scaffolded discussion templates to analyze and answer questions. Fluency may be monitored with each passage. 55

62 This part of the Tool Kit includes: y y y Student copy of passages and comprehension questions Scaffolded discussion templates Passages that are used for monitoring fluency Student Placement and Usage: For students scoring below the 30th percentile on the Maze and/or Word Analysis tasks, reading intervention teachers may use a timed Word Placement List to place students into a set of instructional passages that a student will be able to read with at least 90% accuracy. In this way, decoding can be controlled so that comprehension can be the focus of instruction. The teacher may select one passage from the set to use with a student or group of students. Accompanying these passages are Scaffolded Discussion Templates that the teacher can use to scaffold students understanding of the passage. Additionally, these passages have been equated for difficulty and, therefore, may be used to monitor fluency. Equated fluency scores are available in a look-up table at Students should be placed into passages according to their fluency performance on the word placement list. Students that meet or exceed the 40th percentile for their grade level will be placed into one set of passages, while students who are below the 40th percentile will be placed into another set of passages from a placement table available at The passages listed in this table are equated for difficulty. Template format: The two above-mentioned approaches are incorporated into each scaffolded discussion template. There are eight templates per passage (i.e., one template for each comprehension question). Question and choices: Teacher explains: The QAR type (e.g., Right There, Think and Search, Author and Me, On Your Own) is discussed and explained in this section of the template. In addition: The teacher explains that it may be necessary to go back to the question and get further information. Signal words (e.g., most important, DIFFERENT) are used to encourage the student to focus on essential words in the question. Put the two together: The teacher explains how information from the first two sections is combined to help determine the answer. 56 Alternate thinking process: This approach can be used separately or in conjunction with the QAR portion of the scaffolding. A hypothetical question that a student might ask is posed. This is followed by a possible method of support that the teacher can provide State of Florida, Department of Education. All Rights Reserved.

63 Word Placement Procedure to Monitor Fluency Using the Instructional Level Passages with Scaffolded Discussion Templates These templates can be used to scaffold student thinking about comprehension questions and to monitor fluency through equated passages according to performance on the word placement list. Additional Materials QAR Teacher Template Placement Word List, blank cover sheet, stopwatch. Practice None Directions 1. Place the Task Card in front of the student. 2. Cover all of the words with a cover sheet while reading the script. 3. Uncover the words. 4. Instruct the student to read the word list (read script). 5. Mark all errors made (see scoring rules). 6. If the student hesitates for 4 seconds, mark the word incorrectly and say, "Next Word." 7. Time the student for 45 seconds and then say, "Stop." 8. Draw a line after the last word read. If the student completes the word list before 45 seconds has elapsed, record the time in the space provided. 9. If the student indicates that s/he cannot read any more words before 45 seconds has elapsed, ask the student to look over the list to see if there are any other words s/he can read. If not, end testing. Tip Only mark the words the student misses or skips with a slash as the typical word reading pace does not allow time to write 0 or 1 during administration. Teacher Script I want you to read some words as fast as you can. Say the words so I understand you. When you get to the bottom of a row, quickly go to the top of the next row. If you don't know a word, skip it. Ready? (Remove the cover sheet.) Start here (point to the first word). Scoring Score correct responses = 1. Score incorrect responses = 0. Placement Students should be placed into passages according to their fluency performance on the Word Placement List. Students that meet or exceed the 40th percentile for their grade level will be placed into one set of passages, while students who are below the 40th percentile will be placed into another set of passages from a placement table available at The passages listed in this table are equated for difficulty. 57

64 1 grabbed 2 wheels 3 climb 4 hidden 5 loose 6 forests 7 sideways 8 smooth 9 hospital 10 closely 11 squeeze 12 favorite 13 correct 14 nearby 15 mirror 16 fifteen 17 newspapers 18 fuel 19 carelessly 20 surprise 21 crowded 22 landscape 23 predicted 24 frightened 25 anybody 26 hamburger 27 continue 28 balance 29 references 30 treatment 31 dangerous 32 information 33 discover 34 conducting Placement Word List for Scaffolded Discussion Templates Score 0, 1 35 volunteers 36 announce 37 pollution 38 disappeared 39 entirely 40 qualified 41 knowledge 42 fierce 43 gradually 44 dissolve 45 ordinary 46 regular 47 serious 48 recently 49 beneath 50 furious 51 accounted 52 generating 53 lectures 54 division 55 isolation 56 ordered 57 escape 58 provided 59 fundamental 60 results 61 citizenship 62 liquid 63 urgent 64 quantity 65 reservations 66 pleasant 67 century 68 dissatisfied Score 0, 1 69 miserable 70 adequately 71 offensive 72 muscular 73 neutral 74 valuable 75 figures 76 chemical 77 foreign 78 poisoning 79 authorized 80 explained 81 thoughtfully 82 unusually 83 legitimate 84 vulnerable 85 cooperate 86 advancement 87 prosperous 88 specifically 89 accumulated 90 invariably 91 collapsed 92 unconscious 93 professional 94 domesticated 95 acquaintance 96 pneumonia Score 0, 1 Total Correct / State of Florida, Department of Education. All Rights Reserved.

65 grabbed anybody beneath neutral wheels hamburger furious valuable climb continue accounted figures hidden balance generating chemical loose references lectures foreign forests treatment division poisoning sideways dangerous isolation authorized smooth information ordered explained hospital discover escape thoughtfully closely conducting provided unusually squeeze volunteers fundamental legitimate favorite announce results vulnerable correct pollution citizenship cooperate nearby disappeared liquid advancement mirror entirely urgent prosperous fifteen qualified quantity specifically newspaper knowledge reservations accumulated fuel fierce pleasant invariably carelessly gradually century collapsed surprise dissolve dissatisfied unconscious crowded ordinary miserable professional landscape regular adequately domesticated predicted serious offensive acquaintance frightened recently muscular pneumonia 59

66 Materials Lexiled Text Passages Directions Use these passages to conduct detailed observations of students approach to and use of comprehension strategies when reading connected text. Independent Level Reading Comprehension passages with comprehension questions (found in Tool Kit) Directions Use the student s Lexile score level from the Reading Comprehension Screen to match him or her to a passage. If a student is between two passages, use the passage with the lower Lexile score level. The student will read the passage and then answer comprehension questions about the passage. Students may read these passages silently in groups, working collaboratively to write responses to comprehension questions, or may discuss responses orally in group discussion. NOTE: These passages have not been precisely equated for difficulty within grade level, so they should not be used for the progress monitoring of oral reading fluency. Suggested Teacher Script I would like you to read a passage and then answer some questions. As you are reading, think about how you are making sure you understand the meaning of the passage. What strategies are you using? I will then ask you some questions about what you have read State of Florida, Department of Education. All Rights Reserved.

67 Academic Word Inventory Use this informal inventory to collect additional information regarding student academic word knowledge. Materials Provided One student academic word list for each grade, grades 3-12 (word lists are numbered according to grade level, e.g., Word list 3= 3 rd grade level). Not provided photocopy of student academic word list (from Tool Kit) to record student responses Directions Select the appropriate grade level academic word list to administer to your student. Make a photocopy of that student academic word list to mark incorrect student responses. Place the student academic word list in front of the student and read the Teacher Script. Follow along as the student reads and mark incorrect responses with a slash through the word. Tally total words read correctly. Administer all items. This inventory is untimed. Teacher Script I m going to show you a list of words that I would like you to read aloud. Some may be easy and others may be difficult; do your best to read each word. Ready? Additional Administration Information After the administration of the task, it may be helpful to have a brief discussion with the student to inquire about how s/he felt about the level of difficulty of the task and what s/he was thinking (e.g., strategies used) when faced with an unfamiliar word. Scoring: Score correct responses = 1 Score incorrect responses = 0 Mastery of each word list is 20/25 or 80% correct. 61

68 Phonics Screening Inventory Use this informal inventory to collect additional information regarding student word knowledge. Materials Provided Teacher Administration Sheet for Elementary and Middle/High School, Error Type Analysis Sheet (found in Tool Kit) Not provided Lined paper for students Directions Give each student a lined piece of paper. Have him/her write their name at the top and number it Use the Teacher Administration page to administer the inventory: Say the target word, read the sentence, say the target word again. Pause in between each item to allow student adequate time to write the word. Upon completion of the list, collect each student s paper and analyze incorrect responses using the Error Type Analysis Sheet. One Error Type Analysis Sheet should be used per student. While correcting each student s paper, look across the row and highlight the letter(s) the student misspelled in the appropriate column. Tally the columns to determine the areas of strength and need for the student. Make instructional decisions based on the student s performance. Administer all items. If the Middle/High School Inventory is judged to be too difficult for your student, administer the Elementary Inventory. Teacher Script I m going to say a word, use it in a sentence, and then say the word again. I want you to write down the word on your paper. Ready? Additional Administration Information Students in grades 3-5 should be administered the Elementary Inventory. Students in grades 6-12 should be administered the Middle/High School Inventory. This inventory may be group administered. Scoring: Score correct responses = 1 Score incorrect responses = 0, Use the Error Type Analysis Sheet to categorize student errors. Mastery of each inventory is 20/25 or 80% correct. NOTE: Decisions regarding the organization of letter patterns, roots, and affixes were made to help with the ease of teacher use and to follow typical curriculum sequences of instruction. For example, some words are not fully broken down into their finite roots and affixes as they are not typically taught with this level of detail State of Florida, Department of Education. All Rights Reserved.

69 Instructional Implications 63

70 Using Assessment Results for Guiding Instruction 64 Results from the Florida Assessments for Instruction in Reading provide valuable information when making instructional decisions. The three formal tasks, the Broad Screen/Progress Monitoring tool, the Maze task and the Word Analysis task are given three times a year for different purposes and provide us with different types of information. Ongoing Progress Monitoring and an Informal Diagnostic/Progress Assessment Tool Kit are also part of the assessment system and can be used between assessment periods to provide teachers with additional information to guide instruction. Broad Screen Purpose: Helps us identify students who may not be able to meet the grade level literacy standards at the end of the year as assessed by the FCAT. Information Guiding Instruction: If a student has a relatively high probability of success, it indicates that he/she will most likely be successful on the FCAT when provided with the support and instruction in higher level comprehension skills, reasoning skills, and/or vocabulary that are offered as part of normal instruction in content area classes. If a student has a relatively low probability of success, it means he/she is not likely to be successful on the FCAT without extra support for reading growth during the year, and the TDI tasks are given as a follow up to get more information about the type of help that may be needed. Maze Purpose: Helps us learn how efficiently students can read text silently with comprehension. If a student does poorly on this task, it indicates the presence of more fundamental reading problems in the area of reading fluency and accuracy and/or low level reading comprehension. Information Guiding Instruction: If a student does well on this task, but has a low probability of success from the Broad Screen, this means that he/she is likely to need particular support in the comprehension strategies, vocabulary, and thinking skills required by the FCAT, but likely does not need intensive work to increase basic reading abilities. If a student does poorly on this task, and has a low probability of success from the Broad Screen, this indicates that the student is weak in basic reading proficiency involving either fluent or accurate word reading of text or low level comprehension abilities. For students who struggle on the Maze task, their word-level skills need to be explored more fully using the informal Tool Kit, and remediation focused in areas of weakness. If the intervention teacher were to have the student read a grade level passage orally and did not detect any serious problems reading the words accurately with reasonable fluency, then problems with comprehension monitoring or paying attention to content would need to be further explored. The Maze task simply diagnoses whether a student might have weaknesses in basic reading processes it does not indicate which processes are deficient State of Florida, Department of Education. All Rights Reserved.

71 Word Analysis Purpose: Helps us learn more about a student s fundamental literacy skills particularly those required to decode unfamiliar words and read accurately. Information Guiding Instruction: If a student performs well on the Maze test, but poorly on the Word Analysis test, this would indicate either a special weakness in spelling that does not affect reading, or that the student is able to compensate for poor word reading skill through the application of contextual analysis in reading. Students who are low on both Mazes and Word Analysis may be in need of intensive interventions to improve their basic word recognition skills. Teachers can examine patterns of performance on the Word Analysis test to learn about specific aspects of spelling/word reading knowledge that may require additional instruction. Error Analysis A teacher may think of error patterns in terms of three domains: phonological, morphological, or orthographic. Phonological knowledge is one s awareness of the sound system of language. It has been identified as an essential skill in both reading and spelling development (National Reading Panel, 2000; Treiman & Bourassa, 2000). o A phonological omission error occurs when a phoneme (sound) is not represented by a letter(s) in the spelling of a word (e.g., sp for sip). Morphological knowledge is one s awareness of the units of meaning in language (Goswami, 1998). These units of meaning are commonly referred to as prefixes, suffixes, and root or base words. o A morphological error occurs when: the prefix or suffix is omitted, misspelled, unnecessarily added, or the required change to the base is not spelled accurately (e.g., planng for planning). Orthographic knowledge is one s knowledge of the letter-sound relationships of English and the rules and generalizations of how sounds are represented in print (Cassar & Treiman, 2004; Treiman & Cassar, 1997). In some instances knowledge of the rules and generalizations is not enough. Take for example the word soap. It could also be spelled sope. Both are plausible orthographic spellings and are phonologically accurate, but only one is correct. You can think of orthographic errors in two ways; both ways address using incorrect letter(s) in spelling, but it could be argued that one demonstrates a more sophisticated understanding of spelling. Students with that type of error are making a good choice in representing a legitimate sound; it just happens to be the wrong word in which to use that letter(s). o An orthographic error occurs when: a letter(s) is used to represent a phoneme in a word that is not possible in English orthography; a spelling rule is not applied when required (e.g., bick/back; the letter i never makes the short /a/ sound in English) or a plausible but incorrect representation of the phoneme in that particular word is used (e.g., boyl/boil; oi and oy represent the same sound, but oi is the correct choice in the word boil). 65

72 Ongoing Progress Monitoring Purpose: If intervention teachers would like to monitor the progress of their students more frequently than at the three major assessment points (Fall, Winter, Spring), we recommend the use of the Maze task for this purpose in grades For grades 3 to 5, teachers can choose to use either Oral Reading Fluency probes or the Maze task. Information Guiding Instruction: The Maze task. If the student shows no growth over 2 to 3 ongoing progress assessments, this is a likely signal that the instruction needs to be adjusted if the goal is to close the reading gap for that student. Oral Reading Fluency. If the growth line across several Ongoing Progress Monitoring assessments does not exceed average growth for the grade the student is being progress monitored in, then instructional adjustments to the intervention should be made Informal Diagnostic/Progress Assessment Tool Kit Purpose: These tasks and assessment procedures (as described in the overview) are for use primarily by intervention teachers, but in some cases they can also be used by content area teachers to aid in day-to-day decision making in the classroom. Information Guiding Instruction: Ongoing Progress Monitoring of Oral Reading Fluency: OPM tasks of oral reading fluency consist of multiple passages that represent the tasks in the Targeted Diagnostic Inventory that can be administered between Fall (AP 1), Winter (AP 2), and Spring (AP 3) Assessment Periods to provide more frequent monitoring of student progress in oral reading fluency. OPM tasks will also include equated, short passages for assessing oral reading fluency in one minute. Instructional level passages with Scaffolded Discussion Templates. For students scoring below the 30th percentile on the Maze and/or Word Analysis tasks, reading intervention teachers may use a timed Word Placement List to place students into a set of instructional passages that a student will be able to read with at least 90% accuracy. In this way, decoding can be controlled so that comprehension can be the focus of instruction. The teacher may select one passage from the set to use with a student or group of students. Accompanying these passages are Scaffolded Discussion Templates that the teacher can use to scaffold students understanding of the passage. Additionally, these passages have been equated for difficulty and, therefore, may be used to monitor fluency. Equated fluency scores are available in a look-up table at 66 When monitoring comprehension using either the Lexiled text passages or Instructional Level passages, teachers should consider the multiple aspects of comprehension instruction. It would be helpful to take into account both the strategies to be taught and the type of text on which to use them. The National Reading Panel Report (NRP; NICHD, 2000) found several comprehension strategies to be effective: comprehension monitoring, use of graphic and semantic organizers, question answering, question generation, story structure, summarization, and multiple strategy instruction. In addition to instruction in these 2009 State of Florida, Department of Education. All Rights Reserved.

73 generic comprehension strategies, teachers also need to engage students in discussion about the oral and written language of a specific discipline. Disciplinary literacy refers to the notion that characteristics of texts vary from discipline to discipline that historical texts reflect an historian s interpretation of historical events and are very different from scientific texts that reflect the scientific method and that teachers must be cognizant of the demands of the discourse of each discipline. The following table is based upon the reading framework of the National Assessment of Educational Progress (National Assessment Governing Board, 2008), from which we can describe the text type, text structure, and author s craft of text in each discipline and appreciate that literacy standards vary by discipline. In Reading Next Biancarosa and Snow (2004) suggest that gains in reading and writing in the content areas are best achieved when literacy instruction is embedded in content-area classrooms (Biancarosa et al., 2004). text type text structure literary 2009 NAEP Framework English Mathematics History Science plot, setting, characterization, point of view, verse, rhyme informational or technical, symbolic, diagrams sequence, cause and effect, problem and solution, supporting ideas and evidence, graphical features expository, argumentative, persuasive sequence, cause and effect, problem and solution, author s perspective supporting ideas and evidence, contrasting viewpoints, graphical features Informational or technical, diagrams sequence, cause and effect, problem and solution, supporting ideas and evidence, graphical features author s craft diction, dialogue, symbolism, imagery, irony, figurative language rhetorical structure, examples, logical arguments figurative language, rhetorical structure, examples, emotional appeal rhetorical structure, examples, logical arguments Let s consider what discipline literacy might look like with eighth grade students who are learning about relativity in science. The type of text they would be required to read is informational, describing Einstein s principle of special relativity, and most likely includes some examples to illustrate the key concepts. It may even include a comparison of the principles of relativity of Einstein and another scientist who theorized about relativity (e.g., Galileo, Maxwell, Newton). Arguments would be made in the text highlighting the similarities and differences with supporting evidence or ideas for both scientists. Students would be required to understand the vocabulary (e.g., relativity, magnetism) introduced in the text in addition to its organizational structure. If the students are asked to take the perspective of one of the scientists and argue logically for that individual s point of view, it would require a working knowledge of the way in which scientific arguments are framed. Students are not likely to be taught the text structure of science text in English or math class. Thus it is important for each content-area teacher to help students comprehend and learn how to write based on the literacy expectations of that field. 67

74 Lexiled text passages. For students scoring above the 30th percentile on the Maze task and Word Analysis task but who are still challenged by comprehending grade-level text, teachers may use their Lexile range obtained from the Reading Comprehension Screen to match them to Lexiled passages in the tool kit. Reading intervention or content area teachers may group students assigned to the same passage and ask them to read silently and discuss answers to the comprehension questions. These passages have not been precisely equated for difficulty within grade level, so they should not be used for Progress Monitoring of Oral Reading Fluency. Academic word inventory. This includes lists of academic words that occur with increasing frequency at each grade level to provide the teacher with an estimate of the student s context-free word reading ability. It is to be used by intensive intervention teachers for students scoring low on the Maze task and/or the Word Analysis task below the 30 th percentile. Phonics screening inventory. This includes items and procedures for assessing the specific knowledge required for decoding words phonemically. Letter-sound correspondences, blending ability, skills with complex syllables or different syllable types are examples of the types of skills assessed by this inventory. It is meant to be used by intensive intervention teachers with students who have scored very poorly on the Word Analysis task in the Targeted Diagnostic Inventory State of Florida, Department of Education. All Rights Reserved.

75 Appendix 69

77 Approved Accommodations for the Florida Assessments for Instruction in Reading, Grades 3-12 What are testing accommodations? Accommodations are provided to students with disabilities and/or English Language Learners to assure that valid results are obtained and an accurate estimate of their skill level is determined. These accommodations are specific to this set of assessments. For some students with disabilities, the Florida Assessments for Instruction in Reading may not be an appropriate instrument. For example, it is unlikely that this instrument would be appropriate for students with significant sensory, cognitive, or language deficits. Examiners should ask themselves the question, Is it reasonable to administer these measures to this student? when making a determination regarding assessment of a student with a disability When the assessments are administered in ways different from both a standard administration and the approved accommodations, the administration would be considered a non-standardized administration and the resulting scores cannot be interpreted with the appropriate interpretive and reporting procedures. X indicates that the accommodation is applicable to the task listed at the top of the column. Presentation Reading Comprehension Maze- Word Reading Efficiency Word Analysis Screen magnification may be used. X X X Large print or Braille (if the student has sufficient skills in Braille) versions of the tasks may be used. Note: Some tasks may not be available in Braille, i.e., Maze, Word Analysis. X Not Available Braille Keyboard/Writer X X Students who are deaf or hard of hearing may access the written text directions by clicking on the Read Comments button on the screen. Deaf and hard of hearing students may have a recorder to enter data. Oral or manual interpreters may be used for students who are deaf or hard of hearing to access spoken language. Repeat and clarify general test directions that are presented by the tester or on the board. Tester may prepare student for tasks by explaining the directions that will be included in the web-based application. Once the test begins the student will not be able to stop and ask questions, due to timing issues. X X X X X X Not Available Not Available Not Available Not available Not Available X X X 71

78 72 Presentation Graphic tags are included for all images that are in the Reading Comprehension Task to assist students who are visually impaired. Responding The test can be completed using the mouse, keyboard, or assistive technology that uses either device. Note: Assistive technology that reads or spells any of the task items for student may not be used. Reading Comprehension 2009 State of Florida, Department of Education. All Rights Reserved. X Maze- Word Reading Efficiency Not Appropriate Word Analysis Not Appropriate X X X Extended Time X Not Available Not Available Setting and Tester Adaptive furniture/equipment is appropriate if this is a typical accommodation for the student in the X X X learning environment. Alternate setting is allowable, e.g., special room with minimal distractions, complete quiet or X X X specialized lighting. Student may be monitored while taking the online assessments by someone familiar with the student s language and communication mode and with X X X whom the student is comfortable. Student may be monitored while taking the online assessments by a professional with relevant specialized training such as an occupational therapist for a student with motor difficulty. X X X Appropriate training on the assessments is essential. Repeating the assessment under different conditions or with different testers is an allowable accommodation if it is determined that this accommodation may potentially impact a students X X X performance (e.g., the student was obviously upset, ill, or very uncomfortable in the testing environment). Assistive Devices Student may use amplification device(s). X X X Student may use augmentative communication device. X X X Note: If a cell indicates Not Available, the task is not appropriate for students who are visually impaired or d eaf / hard of hearing. To make the tasks available for these students would change the constructs of the tasks and the resulting scores would become invalid.

79 Books and Guidance Documents: Instructional Resources Boardman, A. G., Roberts, G., Vaughn, S., Wexler, J., Murray, C. S., Kosanovich, M. (2008). Effective Instruction For Adolescent Struggling Readers: A Practice Brief. Portsmouth, NH: Center on Instruction for K-12 Reading, Math, Special Education, English Language Learners, and Science. Retrieved from Center on Instruction. (2009). Adolescent literacy resources: An annotated bibliography: Second edition. RMC Research Corporation, Portsmouth, NH: Author. Retrieved from Florida Online Reading Professional Development (FOR-PD), html Gersten, R., Baker, S. K., Shanhan, T., Linan-Thompson, S., Collins, P., & Scarella, R. (2007). Effective Literacy and English Language Instruction For English Learners in the Elementary Grades: A Practice Guide (NCEE ). Washington, DC: National Center for Education Evaluation and Regional Assistance, Institute of Education Services, U.S. Department of Education. Retrieved from Heller, R. & Greenleaf, C. (2007). Literacy Instruction in the Content Areas: Getting to the Core of Middle and High School Improvement. Washington, DC: Alliance for Excellent Education. Retrieved from Henry, M. K. (2003). Unlocking Literacy. Baltimore: Brookes. Kamil, M. L., Borman, G. D., Dole, J., Kral, C.C., Salinger, T., & Torgesen, J. K. (2008). Improving Adolescent Literacy: Effective Classroom and Intervention Practices: A Practice Guide (NCEE # ). Washington, DC: National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences, U.S. Department of Education. Retrieved from McPeak, L., & Trygg, L. (2007). The Secondary Literacy Instruction and Intervention Guide. Mill Valley, CA: Stupski Foundation. Retrieved from Torgesen, J. K., & Miller, D. H. (2009). Assessments to Guide Adolescent Literacy Instruction. Portsmouth, NH: RMC Research Corporation, Center on Instruction for K-12 Reading, Math, Special Education, English Language Learners, and Science. Retrieved from Torgesen, J., Houston, D., & Rissman, L. (2007). Improving literacy instruction in middle and high schools: A guide for principals. Portsmouth, NH: RMC Research Corporation, Center on Instruction for K-12 Reading, Math, Special Education, English Language Learners, and Science. Retrieved from Torgesen, J. K., Houston, D. D., Rissman, L.M., Decker, S. M., Roberts, G., Vaughn, S., Wexler, J., Francis, D. J., & Rivera, M.O. (2007). Academic literacy instruction for adolescents: A guidance document from the Center on Instruction. Portsmouth, NH: Center on Instruction for K-12 Reading, Math, Special Education, English Language Learners, and Science. Retrieved from 73

80 Websites: Adolescent Literacy, Center on Instruction, Council of Chief State School Officers, Florida Center for Reading Research, Florida Department of Education, Florida Literacy and Reading Excellence Center (FLaRE), Florida Response to Intervention, Just Read, Florida!, LD Online, LEaRN (Literacy Essentials and Reading Network), Reading First Professional Development (RFPD), State of Florida, Department of Education. All Rights Reserved.

81 References Armbruster, B. B., Lehr, F., and Osborn, J. (2001). Put Reading First: The Research Building Blocks For Teaching Children to Read. Washington, DC: Partnership for Reading. Biancarosa, G., & Snow, C. E. (2004). Reading next A vision for action and research in middle and high school literacy: A report to Carnegie Corporation of New York. Washington, DC: Alliance for Excellent Education. Cassar, M., & Treiman, R. (2004). Developmental variations in spelling: Comparing typical and poor spellers. In C. A. Stone, E. R. Silliman, B. J. Ehren, & K. Apel (Eds.), Handbook of language and literacy: Development and disorders (pp ). New York: Guilford Press. Goswami, U. (1998). The roles of analogies in the development of word recognition. In J. L. Metsala & L. C. Ehri (Eds.). Word recognition in beginning literacy (pp ). Mahwah, NJ: Lawrence Erlbaum Associates. National Assessment Governing Board (2008). Reading framework for the 2009 National Assessment of Educational Progress. Washington, DC: U.S. Department of Education. National Reading Panel (2000). Teaching children to read: An evidence-based assessment of the scientific research literature on reading and its implications for reading instruction. NIH Publication No Washington, DC: National Institute of Child Health and Human Development. Raphael, T. E. (1984). Teaching learners about sources of information for answering comprehension questions. Journal of Reading, 27(4), Raphael, T. E. (1986). Teaching question answer relationships, revisited. The Reading Teacher, 1986, 39, 6, Feb, 39(6), Raphael, T. E., & Au, K. H. (2005). QAR: Enhancing comprehension and test taking across grades and content areas. Reading Teacher, 59(3), Raphael, T. E., & Pearson, P. D. (1985). Increasing students awareness of sources of information for answering questions. American Educational Research Journal, 22(2), Snow, C. E., Griffin, P., and Burns, M. S. (Eds.) (2005). Knowledge to Support the Teaching of Reading: Preparing Teachers for a Changing World. San Francisco: Jossey-Bass. Treiman, R., & Bourassa, D. C., (2000). The development of spelling skill. Topics in Language Disorders, 20(3), Treiman, R., & Cassar, M. (1997). Spelling acquisition in English. In C. A. Perfetti, L. Rieben, & M. Fayol (Eds.), Learning to spell: Research, theory, and practice across languages (pp ). Mahweh, NJ: Erlbaum. Wayman, M., Wallace, T., Wiley, H., Ticha, R. & Espin, C. (2007). Literature synthesis on curriculumbased measurement in reading. The Journal of Special Education, 41,