The DLM Assessment System: Planning for 2014-15 Implementation November 13 & 14, 2014
Today s Topics 1. Overview of the DLM system and year end testing model 2. 2014-2015 from the teacher s perspective 3. District support for implementation 4. Other topics as requested
OVERVIEW OF ASSESSMENT SYSTEM
A Portion of the Math Map
Learning Map Claims Conceptual Areas Essential Elements (and other nodes)
DLM Essential Elements Are the target for the grade level Reduced depth, breadth, complexity Provide appropriate level of rigor and challenge Focus on the skills (with multiple means of demonstration) Are not functional or pre-k skills or instructional descriptions
Example for English Language Arts Common Core State Standard RL.6.2 Determine a theme or central idea of a text and how it is conveyed through particular details; provide a summary of the text distinct from personal opinions or judgments. Essential Element EE.RL.6.2 Determine the theme or central idea of a familiar story and identify details that relate to it.
Identify two related points the author makes in an informational text
Testlets in Linkage Levels Connect the map Initial Precursor Behavior to the items developed. Testlet IP Distal Precursor Proximal Precursor Behavior Behavior Testlet DP Testlet PP Target Successors Behavior Behavior Testlet T Testlet S
Fractions M.EE.3.NF1-3 Differentiate a fractional part from a whole 10
Assessment Design Testlets Usually assess 1 EE / linkage level 3-5 items Engagement activity ELA: Text or writing task Math: Context at beginning, carried throughout Teacher-administered or computeradministered (MC, sorting, matching)
Special user interface designed for DLM Customized presentation through Personal Needs & Preferences (PNP) profile
Sample Testlets ELA: RI 5.8: Identify the relationship between a specific point and supporting reasons in an informational text Target Level Node: Same as EE Math: 7.NS.2.a: Solve multiplication problems with products to 100 Initial Precursor Level Nodes: Recognize set, recognize separateness
Two Levels of Writing Assessments Conventional Writing Using orthography to communicate Selecting topics Generating and communicating facts, ideas and details in writing Emergent Writing Behaviors that precede conventional writing Communication about choices and information Cognitive processes that are precursors to conventional writing
Example Task/Rating in an Emergent Writing Testlet Screen 1 SAY: What do you want to write about? Choose one. WAIT AND OBSERVE: Give the student time to choose one of the objects.
Example Task/Rating in an Emergent Writing Testlet Screen 2 Choose the highest level that describes your observation. Makes a choice between objects. Attends to objects but does not make a choice. Seeks objects but does not make a choice. Attends to other stimuli. No response.
Example Task/Rating in an Conventional Writing Testlet Screen 1 SAY: What other facts, ideas or details can you write about [topic]? WAIT: Give the student time if needed to think about additional facts, details or other information about the topic. SAY: Write about them. WAIT AND OBSERVE: Give the student time to write facts, details or information about the topic.
Example Task/Rating in an Conventional Writing Testlet Screen 2 Choose the highest level that describes your observation. Independently wrote 3 or more facts, ideas or details related to the topic Independently wrote at least 2 facts, ideas or details related to the topic Wrote one fact related to the topic Wrote multiple facts, ideas or details unrelated to the topic. Wrote a single fact, idea or detail unrelated to the topic.
ACCESSIBILITY
Accessibility Accessible Content Personal Learning Profile Technology
Accessible Content Testlet levels Vocabulary Multiple and alternate pathways Special item writing guidelines Careful attention to unnecessary complexity
Personal Learning Profile Personal Needs and Preferences (PNP) Display Language & Braille Audio & Environment First Contact Communication Academics Sensory characteristics Motor characteristics Computer access
Technology Special user interface designed for DLM Dynamic routing
Accessibility Manual 1. Include Eligible Students in the DLM Assessment 2. Learn About the DLM Accessibility Features: What Does DLM Provide? 3. Discuss and Select Appropriate Supports and Tools: Considerations for IEP Teams 4. Enter Appropriate Supports into the DLM System 5. Prepare for the Assessment: Using the Chosen Accessibility Features 6. Evaluate the Accessibility Features Used After the Assessment
1. Supports activated by PNP A. Magnification B. Invert Color Choice C. Color Contrast D. Color Overlay E. Read Aloud with highlighting Text to Speech text only text & graphics graphics only nonvisual
2. Supports Requiring Additional Materials/Tools F. Uncontracted Braille G. Single-switch system/pnp enabled H. Two switch system I. Administration via ipad J. Adaptive equipment used by student K. Individualized Manipulatives
3. Supports Provided Outside the System L. Human Read Aloud M. Sign interpretation of text N. Language translation of text O. Test administrator enter responses for student P. Partner Assisted Scanning
Personal Needs & Preferences (PNP) Profile Display Enhancements Language & Braille Audio & Environment Other Supports
Students with Blindness/Visual Alternate forms Impairments Content would normally be represented on computer but translated into teacheradministered testlet Braille Alternate pathways
Braille Very small number of DLM students use Braille Forms will be available: Beginning spring 2015 operational window Only in upper linkage levels Delivery method still under evaluation by state partners
Hardware/Software No software costs (KITE is free) PNP features built in Use devices already available (interactive whiteboard, 1-switch or 2-switch systems, headphones, etc.) Option for test administrator to enter responses if there is a gap
HOW THE YEAR END MODEL WORKS
Year End Assessment Model Optional Instructionally- Embedded Optional Instructionally Embedded Testlets can be delivered for select Essential Elements System recommends linkage level; teacher can accept or override Student performance does not contribute for summative purposes Required Spring Testing Testlets cover the whole blueprint Dynamic routing supports adaptivity between testlets (system determines linkage levels) Scores are used for summative purposes
Content of the Assessment Grades 3-8 and high school English Language Arts (ELA) Mathematics Blueprints: Consortium approved a subset of Essential Elements Consortium set minimum requirements for breadth of coverage
Test Delivery Optional Instructionally Embedded One testlet per EE chosen in ITI Teacher chooses linkage level Each testlet assigned separately Test administrator schedules sessions within testing windows Spring (End of Year) 4-7 testlets per subject depending on grade Entire blueprint covered System chooses linkage levels Each testlet assigned separately Test administrator schedules sessions within window
Testlet Delivery System has testlets available at all 5 linkage levels for every part of the test Part 1 IP Part 2 IP Part 3 IP Part 4 IP Students take one testlet from one level for each group DP PP DP PP DP PP DP PP System determines which level to deliver to the student T S T S T S T S
Testlet Delivery Part 1 Part 2 Part 3 Part 4 IP IP IP IP DP DP DP DP PP PP PP PP T T T T S S S S
Testlet Delivery Part 1 Part 2 Part 3 Part 4 IP IP IP IP DP DP DP DP PP PP PP PP T T T T S S S S
Testlet Recommendations & Assignments
Performance Profile Part 1: Overall Results
Performance Profile Part 2: Conceptual Areas
Performance Profile Part 3: Narrative
THE KITE SYSTEM
Installed on testing devices Student login credentials
Website access (use Firefox) Staff login credentials
2014-15 FROM THE TEACHER S PERSPECTIVE
Key Steps 1. Training & PD 2. Eligibility and accessibility decisions 3. Manage student data in Educator Portal 4. Select content; provide instruction 5. Optional: prepare for & deliver instructionally embedded assessments 6. Prepare for & deliver testlets during spring window 7. Plan for next year
Required Training vs. Professional Development Training Multiple modules Available in self-directed and facilitated formats (states/districts decide which to use) Covers critical content for managing and delivering the assessment Required for all test administrators No access to student logins without it PD Multiple modules Available in self-directed and facilitated formats (states/districts decide which to use) Covers a variety of topics to support instruction in academics States determine what is required/optional
Other Resources for Test Administrators Test Administration Manual With checklist Accessibility Manual Information about EEs and levels Access to released testlets and practice activities Educator resource page with supplemental info & videos
Management of Student Records Accuracy of roster Correct students, correct information First Contact PNP: Accessibility features discussed with IEP teams Documented in IEP Instructional plans (if using ITI)
Verify Students & Rosters
First Contact
PNP
Instructional Tools
DISTRICT SUPPORT FOR IMPLEMENTATION
DLM-Defined Roles Data Steward Technical Liaison Assessment Coordinator Test Administrator (teacher, educator) Individuals may fill more than one role.
Some States Have Other Roles Regional technical assistance provider Mentor/facilitator Vendor (other support services) Others? How do these fit with DLM roles?
Data Steward Responsibilities Gather, validate, and upload teacher, student, and roster data into Educator Portal Update records as needed
Technical Liaison Responsibilities KITE Client Troubleshooting Educator Portal Troubleshooting CSV files Whitelisting *.cete.us LCS, Local Caching Server
Assessment Coordinator Responsibilities Manage district/school staff roles Develop and implement training plan Develop and implement communication plan Monitor / support test preparations and administration
Checklist Categories: Before beginning assessment Information/resources Steps to take During (Optional) instructionally embedded assessment Spring assessment Prepare for next year
Before Assessment Manage district/school staff roles and responsibilities. Identify test administrators/examiners Identify a Technical Liaison Share roles/responsibilities and timelines with staff Share DLM policies and procedures
Before Assessment Prepare for assessment implementation Disseminate Technical Liaison contact information Encourage sign up for test notifications Establish communications between Data Steward, Technical Liaisons, and DLM. Develop communication channels with district personnel (updates, etc.) Monitor creation of Educator Portal user accounts Follow up with test administrators Establish a test security plan Troubleshoot issues
User Account Creation Must be on the list and active
Common Roles in Educator Portal District Test Coordinator Teacher Building Test Coordinator Building & District Users
Assigning Permissions in EP State District District Building
Unusual Cases Contract teachers Out of district students Charter schools
Before Assessment Develop and implement a comprehensive training plan OR implement your state s training plan for district staff and test administrators. Review the checklists (TL & TA) Encourage Technical Liaisons to complete training Deliver required test administration training Monitor educator completion of required training Identify additional training needs Monitor participation in DLM professional development
Required Training
Monitoring Completion of RT Now Participant can take screen shot of transcripts area State PD administrator can look up individuals State PD administrator can run PD completion report, share with districts Starting in January New District PD administrator role look up records, run reports
Before Assessment Monitor and support test preparations. Remind Technical Liaisons and Test Administrators about important deadlines. Monitor identification of students for DLM Ensure every student in the system is assigned to a teacher Monitor completion of Access (PNP) Profile and First Contact Assist test administrators with technology preparations Support test environment preparation Manage student usernames and passwords for KITE
Monitoring in EP
Before Assessment Develop a plan to facilitate communications with parents and district staff. Consider what parents and district staff will need and want to know about DLM assessments in 2014-15. Determine the frequency of communications to various audiences. Deliver communication to various audiences on the timeline in your communication plan. Relay new information from DLM and your state and district to appropriate audiences as needed.
Before the Spring Window Support Test Administrators in rechecking student enrollment, demographic information, PNP settings, and First Contact responses.
Checklist Categories: Before beginning assessment Information/resources Steps to take During instructionally embedded assessment Spring assessment Prepare for next year
Spring Window Monitor and support test administration. Facilitate communications between DLM staff and district staff during testing window(s). Obtain and deliver Braille forms as needed. Monitor test activity, including test completion prior to end of testing window. Serve as the first point-of-contact for teachers, answering questions about the assessment. Troubleshoot issues as they arise. Serve as the point-of-contact to the DLM help desk.
Monitoring Available in January: Report showing number of testlets completed and in progress vs. what is intended
Checklist Categories: Before beginning assessment Information/resources Steps to take During instructionally embedded assessment Spring assessment Prepare for next year
Resources & Next Steps Bookmark your state s page on the DLM website. Use the resources on your state s page. Activate your Educator Portal account by following the instructions in the KITE activation email.* Complete the Security Agreement in Educator Portal profile.* Sign up for DLM test notifications during the year.
Website Resources
Help Desk Support Three tiers of support for local users Information published in the front of all manuals Toll-free phone number Extended hours during testing windows Email with ticketing system Answers customized per testing model & for unique state policies
Communication KITE system Status on website Testing updates subscription Special email to states Other issues Special email to states Additional postings, announcements as warranted
OTHER DISCUSSION TOPICS
What questions do you have?
THANK YOU! For more information, please contact: dlm@ku.edu or Go to: www.dynamiclearningmaps.org For Professional Development, contact: dlm@unc.edu The present publication was developed under grant 84.373X100001 from the U.S. Department of Education, Office of Special Education Programs. The views expressed herein are solely those of the author(s), and no official endorsement by the U.S. Department should be inferred.