LevelSet: Online Lexile Assessment Accurately match students to instructional text without taking time away from instruction Achieve3000 s LevelSet, an online assessment that utilizes the Lexile Framework, offers a scientific means of matching students to informational texts. Developed by Achieve3000 in conjunction with MetaMetrics Inc., the renowned psychometric research team best known for the Lexile Framework for Reading, this assessment is delivered via the Internet and is designed to work hand-in-hand with Achieve3000 s differentiated reading solutions KidBiz3000, TeenBiz3000, Empower3000, and Spark3000. Available in both English and Spanish, LevelSet is the only assessment of nonfiction Lexile reading abilities. It enables your students to experience expository content that s tailored to their individual reading levels without extensive, timeconsuming tests. Students simply log onto Achieve3000 Solutions using their pre-assigned user names and passwords. They read a series of approximately 30 paragraph-long passages and answer a clozestyle (fill in the blank) question about each one. The placement test usually takes just one class period (30-40 minutes) and results in an individual Lexile placement score for nonfiction text for each child. Depending on the student, some educators prefer completing the test in several sessions over several days. There s only one way to help students read and write better: By teaching them one-on-one at their own level. Achieve3000 provides the first Webbased, individualized instruction solutions for reading and writing that reach every student at his or her Lexile level. Powered by a proprietary software engine that distributes assignments and assessments to the entire class, but tailors them according to each student s reading level, Achieve3000 enables teachers of grades 2-12 to move their students up surely and steadily, level by level. Achieve3000 Solutions extend teachers reach without increasing their workloads or time demands. Perhaps most important of all, Achieve3000 Solutions are scientifically proven to accelerate reading comprehension, writing proficiency, and performance on highstakes tests. LevelSet test scores are delivered to teachers and administrators in real time, so school and district decisionmakers can immediately see how students have placed into the program. The results of this summative assessment are applied immediately to each student s academic profile so they can receive reading passages that are precisely and scientifically matched to their individual reading levels. This enables Achieve3000 to do what it does best: deliver reading assignments and assessments that are tailored to students reading levels, a proven strategy for improving their performance on high-stakes tests. A summative measurement of student progress LevelSet is administered up to three times yearly, starting with an initial assessment to establish a baseline score (based on the Lexile Framework) at the beginning of the school year, followed by an interim assessment halfway through the school year for eligible schools and students, and finally a post-assessment at the end of the year.
Page 2 Frequently Asked Questions What is the Lexile Framework? The Lexile Framework is a scientific approach to reading and text measurement that has become the most widely adopted reading measure in use today. Developed by MetaMetrics, beginning with a grant from the National Institute of Child Health and Human Development in 1984, Lexile measures are the result of more than 20 years of ongoing research. The Lexile Framework is unique because it measures text difficulty and reader ability on the same scale. This makes for an exceptionally effective matching of readers to appropriate texts. The Lexile Framework has been aligned to many national and state norm-referenced assessments, including the SAT-10, SAT-9, SDRT-4, MAT-8, TerraNova Assessment Series (CTBS/5 and CAT/6), and the Iowa Tests (ITBS and ITED), as well as many state tests. In 2003, the National Center for Education Statistics (NCES) chose the Lexile Framework as its own measure of text difficulty. Lexile measures are based on the relationship between two well-established predictors of how difficult a text is to comprehend: semantic difficulty and syntactic complexity. In the Lexile system, semantic difficulty is measured using the mean log word frequency, which is the logarithm of the number of times a word from the text appears in each five-million words of a corpus of nearly 600- million words. Syntactic difficulty is a measure of the length of the sentences in a passage. Long sentences are likely to contain more clauses and therefore communicate not only more information and ideas, but also an interrelationship between them. Researchers also speculate that longer sentences require the reader to retain more information in short-term memory. The Lexile equation takes into account both of these measurements of word frequency and sentence length, resulting in a measure of reading difficulty, expressed as a Lexile. Why did Achieve3000 choose the Lexile Framework? Achieve3000 chose to use the Lexile Framework after an intensive study of many readability measures. Since the Lexile Framework has been extensively validated by research including a 15-year study funded by the National Institute of Health Achieve3000 and the education community are confident in its superior ability to match readers to text. A key advantage to using the Lexile scales, as opposed to other readability measures, is that the Lexile Framework evaluates both text difficulty and reader ability using the same scale. This means that the ability to comprehend and the material being comprehended are evaluated using the same criteria, lending it greater effectiveness in terms of matching readers with appropriate texts. How many versions of the test are available, and which one is administered to my students? Achieve3000 provides five versions of the summative assessment for students at a range of reading levels. All five versions are available in both English and Spanish. For the pre-test, Standard Edition users automatically receive the appropriate assessment based on their grade level*. Pretest placement is differentiated for Special Editions students: Intervention and Language Edition users start with a test suited for students reading two grades below their grade level; Enrichment Edition users start with a test suited for students reading two grades above their grade level. However, teachers can manually set the test level if desired. For the interim and post-test, all students are placed in the appropriate level of the test based on their current Lexile (rather than grade) level. To ensure an accurate measure of student ability, all tests self-adapt as the student progresses through the items.
The following table shows which LevelSet test level is given to students for the pre-, interim and post-tests: Pre-Test (based on grade level) Interim and Post-Test (based on Lexile level) Version 1 1-4 th grade 580L and below Version 2 5 th and 6 th grade 585L to 780L Version 3 7 th and 8 th grade 785L to 880L Version 4 9 th and 10 th grade 885L to 980L Version 5 11 th -13 th grade 985L and above Page 3 *For Spark3000 users, the system automatically delivers the test at a 8 th grade reading level. If an instructor would like to change the default level for a Spark user, he or she can change the student s reading level. Is LevelSet considered a norm-referenced test or a criterion-referenced test? LevelSet is used in a criterion-referenced manner, since the purpose of the test is to match students with appropriately-leveled reading materials. In addition, the multiple choice activities on Achieve3000 are used in a criterion-reference manner, measuring comprehension of the materials. We do not report any comparisons to a normative population, though the Lexile measures are based on national norms. Is LevelSet available in other languages? LevelSet is currently available in English and in Spanish. The Spanish version is delivered automatically to students using the Full Spanish version. (Note that students must be assigned to the Language Edition in order to have the Full Spanish language setting.) When are the tests administered? LevelSet is delivered once at the beginning of the school year, again halfway through for eligible schools and students, and finally, at the end of the school year. Once delivered, LevelSet displays automatically and students cannot enter the Achieve3000 program until it is completed. Dates for LevelSet administration in the 2011-2012 school year: Pre-test The pre-test is available the first time a student logs in, and remains available until the student completes the test. Interim-test For eligible schools that are not enrolled in the auto-adjust pilot program, the interim test will be delivered between December 12, 2011 and February 20, 2012. You will be notified when your school becomes eligible, and the test will be delivered to students two weeks later. The test will remain available until students complete it. For schools that are enrolled in the auto-adjust program, students who scored below 50L on the LevelSet pre-test and currently have Lexile levels below 50L are eligible for the interim test. The interim test will be delivered to students on December 12, 2011, and you will be notified two weeks prior to this date. Teachers can change the test date on a class level. If you would like to make changes to the test date on an individual level, please contact Customer Support. Administrators can request a school-wide change by contacting Customer Support. Post-test Administrations begin on April 16, 2012.
What makes a school eligible for the interim test? NOTE: For schools enrolled in the auto-adjust pilot program, only students with Lexile scores below 50L will be eligible for the interim test. For non-pilot schools, in order to be eligible for the interim test, 50% of students in your school who completed the LevelSet pre-test must have completed at least 40 Multiple Choice News activities. If 50% of students in your school who took the LevelSet pre-test reach the 40-activity usage goal at any time during the testing window, the interim test will be scheduled for delivery to all students in your school. School staff and students will be notified two weeks prior to the test. Administrators should use the report titled Has my school met interim test eligibility requirements? to measure progress towards eligibility for the interim test. Can we request a special date for these tests? To request an alternate test date for your students, please contact Customer Support at www.achieve3000.com/support. Page 4 How are the tests scored? Achieve3000 calculates students scores automatically, in real time, so results are available immediately when the student completes the test. Scores are recorded in an online system so teachers and administrators can easily access test results. An alert on teachers home pages will be displayed as soon as any student has completed LevelSet. The alert will remain available for two weeks following the test. (To access student scores, log into your Teacher s Edition, click Admin and then the + sign to expand Performance Reports, and refer to the report titled How has Lexile performance changed over time Summative LevelSet? For auto-adjust pilot users, select the report titled: How has Lexile performance changed over time? ). How is student performance reported? The Lexile method of scoring differs from other measures. Achieve3000's reports always "translate" each student's Lexile score to an approximate grade-equivalent reading level. For example, if a student scores a 600L, his/her score is translated to a 3 and he/she receives reading materials appropriate for a student reading at approximately a third grade level. Students who score above a 1350L are shown in our program as AR (advanced reader). They receive content that is targeted to their advanced reading abilities. Students who score below 290L are shown in our program as BR or PRE level readers. While all BR and PRE students receive the same content within Achieve3000, their Lexile levels can range from -600L to 289L. Teachers can refer to students Lexile levels to indicate where along the continuum of early reading skills the student lie
Page 5 The chart that follows can help you understand whether your students are considered on track for college and career readiness as defined by the Common Core State Standards (CCSSO and NGA, ELA Appendix A, 2010). There is overlap across the grade levels. For example, a fourth grade student who scores a 785L is considered on track, but he/she is at the lower end of the continuum. A third grade student who scores a 785L is on track as well, but is at the higher end of the continuum (on track for grade 3 ranges from 590L-790L). Grade On Track 1 220L 500L 2 450L 620L 3 550L 790L 4 770L 910L 5 865L 980L 6 955L 1035L 7 1005L 1085L 8 1045L 1155L 9 1080L 1230L 10 1110L 1305L 11/12 1215L 1355L * Note that any student in grade 1 and above who is reading below 200L is not on track for college and career. How does Achieve3000 use students Lexile levels? Achieve3000 uses the student s Lexile level to scientifically match him/her to an appropriatelyleveled passage. Each reading passage is assigned a discrete Lexile score. Each student, in turn, is assigned a Lexile level. (This level is initially determined by the LevelSet test results and it is adjusted by the system as students work through the program. It can also be changed independently by the teacher based on information from other measures.) Using the Lexile system to match readers to text leads to an expected 75 percent comprehension rate not too difficult to be frustrating, but difficult enough to be challenging and to encourage reading progress. In order to ensure that students are always within this 75 percent comprehension rate, they always receive an article that is no more than 50 points above their Lexile level. In addition, teachers are encouraged to monitor student performance to ensure that students are scoring between 65% and 90% on the multiple-choice activities. If students are scoring outside of this range, their reading level may need to be adjusted. For instructions on adjusting reading levels, see below. In this way, students read about the same topics as others in their class, but the difficulty level of the passage is tailored to their individual Lexile level. If one child has a Lexile level of 400L, while his/her classmate has a Lexile level of 725L, they would both read about the same current news event; but the passages are presented at different Lexile levels, so each child reads a tailor made story. Content in Achieve3000 Solutions generally ranges from 200L to over 1350L. Students in the BR range (1L-289L) will require additional assistance when reading the content. They may get this assistance through use of the audio support or through teacher intervention.
For participants in the auto-adjust pilot program, the system will adjust the student s Lexile level (up or down) based on his/her multiple choice performance once a month, provided that the student has completed at least four Multiple Choice News activities since the last time his/her Lexile level was adjusted and that his/her Lexile level is 50L or higher. Can teachers preview content at different levels? Yes. You can access the daily articles through the content preview. Once at the article, you can easily change from one reading level to another by using the Level drop down on the top navigation bar. Page 6 If I preview the article at reading level 3, will all of my Level 3 students see this article? No. Our system matches students to text based on the Lexile score that they received on LevelSet. So, if two students have the same Lexile score on LevelSet, they will get the same article. But if their scores are different, they could get different versions of the article. The system will send the most closely matched article to each student How do I review students LevelSet scores? An alert ( LevelSet scores are available! ) will display for teachers as soon as any of their students finish LevelSet. The alert will be visible in one of the bulletin boards on the Teacher s Edition homepage and will include a link to a report which shows students LevelSet scores (for non-pilot users, the report is titled How has Lexile performance changed over time (Summative)? ; for pilot users, the report is titled How has Lexile Performance changed over time? ). This report can also be viewed by clicking Admin and expanding Performance Reports. What should teachers do if they have reason to believe that the LevelSet scores are invalid? LevelSet is Achieve3000 s recommended method for determining students Lexile levels. After students have taken LevelSet and completed several multiple choice news activities, however, we recommend that teachers check students LevelSet placement scores and initial activity performance. The LevelSet scores should match what teachers know about each student s reading ability, and the average scores on the first 4-5 multiple choice reading activities should be between 65% and 90%. To review students initial activity results: 1. Log in to your Achieve3000 Teacher s Edition. 2. Click Admin at the top of the screen. 3. Click the + sign to expand Student Work. 4. To the right of News, select Multiple Choice. 5. The page that appears has a series of drop-down boxes. Use the drop-down boxes to customize your report, if desired. 6. Click View Report. 7. A list of activities appears on the screen. By default, the activities are sorted by the ones that were completed most recently. Click the appropriate header to sort the activities by user, activity name, or section. For example, to sort the report by student name, click User. 8. Scan the activity scores. Look for scores between 65% and 90%, on average, for the first few activities. 9. To export the report into an Excel spreadsheet, click Export to Excel at the top of the screen. If you are prompted to open or save, click the desired option. To print the report, click Send to Printer at the top of the screen.
Page 7 If you followed the procedures described above and have identified students with possibly invalid test administrations, you should: 1. Wait for the system to correct for invalid administrations. The system monitors student performance. If a student s performance indicates that the administration of LevelSet was invalid (as demonstrated by either very high or very low activity scores), the system corrects his/her Lexile level up or down. 2. Reset the test for individual students. For non-pilot users, if the system does not correct the test scores, usually because the student s score is not high or low enough to point definitively to an invalid administration, or if you are confident that the test was invalid and don t want to wait for the system, you can re-administer the test. a. Log in to your Achieve3000 Teacher s Edition. b. Click Admin at the top of the screen. c. Click the + sign to expand Assessment Tools. Then click Re-administer LevelSet. d. Check the box to the left of the student name. Click Re-administer LevelSet at the bottom of the screen. The test will appear the next time the student logs in to the program. (Note that the original LevelSet score is replaced by the students new test results. ) 3. Manually adjust the student s Lexile/reading level. Do this within the first two weeks after the test was administered. a. Log in to your Achieve3000 Teacher s Edition. b. Click Admin at the top of the screen. c. Click the + sign to expand User Administration. Then click Edit student or teacher information. d. Select the class and click Submit. e. Locate the student whose reading level you want to change. f. Position the cursor in the Read column and select the desired level. Be sure to set a level that s appropriate for informational text. g. Click Save Changes at the bottom of the screen. (Note that the system still maintains the student s original LevelSet score for future reports). How does Achieve3000 determine students expected post-levelset scores? To view students expected post-levelset scores, non-pilot users can go to Admin, Performance Reports, How has Lexile performance changed over time (Summative LevelSet)? Pilot users should view Admin, Performance Reports, How has Lexile performance changed over time?. These expected gains are based on research with thousands of students at various grade levels and ability levels. To calculate each student s expected score, a MetaMetrics algorithm is used that takes into account the number of months until the post-test as well as the student s grade level and Lexile level. Will my students take any other tests besides LevelSet? Yes. Achieve3000 also provides ongoing formative assessment built right into the instructional routine. Each time they use the program, students read reading-level-appropriate informational (nonfiction) passages and answer a series of multiple choice questions that monitor comprehension, vocabulary mastery, and higher-order thinking skills. These ongoing assessments
help teachers monitor students weekly performance in the program, gauge student progress over time, and assign prescriptive work to target state standards that students have had trouble mastering. Page 8 For pilot users, multiple choice activity scores are used to adjust students Lexile levels, providing up-to-date information about students reading abilities without taking time away from instruction to administer more tests. Once a month, provided that the student has completed four multiple choice news activities since the last time his/her Lexile level was adjusted and that the student s Lexile level is 50L or higher, the system will adjust the student s Lexile level (up or down) based on multiple choice activity performance. Teachers will be alerted via the Teacher s Edition home page one week before the adjustment is to take place, as well as when changes are made to a student s reading level. How much growth should I expect my students to make in one year? Yearly growth varies based on a number of factors, Grade including grade level. Third, fourth, and fifth grade 2 students, for example, are expected to progress 100 3-5 Lexile points in twelve months, while high school students 6-7 are expected to progress only 25-50 Lexile points. The 8-9 chart to the right provides approximate yearly Lexilegrowth expectations. If your students progress more 10-11 Lexile points than is expected for their grade level during one year, they have exceeded the normal reading growth. Expected Yearly Growth 300L 100L 70L 50L 25 Research has found that for every 40 activities students complete on Achieve3000, they make average Lexile gains equivalent to approximately one year s growth. If your students complete 80 activities, you can expect them to make an average of two years worth of gains. Please note: Rate of growth slows down once students make two years worth of progress in a given school year. Test conditions must be identical for pre- and post-test to ensure valid administrations that accurately measure students Lexile levels. See LevelSet Test Tips, below, for more information about how you can ensure valid test administrations. Lexile gains are highest when students apply themselves to their work on Achieve3000. Students who consistently achieve more than 80% on the multiple choice reading activities make significantly higher Lexile gains.
Page 9 LevelSet from Achieve3000: Accurately matching students to instructional text. Helping teachers reach students at their own level.