The DLM Assessment System: Planning for Implementation. November 13 & 14, 2014

Transcription

1 The DLM Assessment System: Planning for Implementation November 13 & 14, 2014

2 Today s Topics 1. Overview of the DLM system and year end testing model from the teacher s perspective 3. District support for implementation 4. Other topics as requested

3 OVERVIEW OF ASSESSMENT SYSTEM

4 A Portion of the Math Map

5 Learning Map Claims Conceptual Areas Essential Elements (and other nodes)

6 DLM Essential Elements Are the target for the grade level Reduced depth, breadth, complexity Provide appropriate level of rigor and challenge Focus on the skills (with multiple means of demonstration) Are not functional or pre-k skills or instructional descriptions

7 Example for English Language Arts Common Core State Standard RL.6.2 Determine a theme or central idea of a text and how it is conveyed through particular details; provide a summary of the text distinct from personal opinions or judgments. Essential Element EE.RL.6.2 Determine the theme or central idea of a familiar story and identify details that relate to it.

8 Identify two related points the author makes in an informational text

9 Testlets in Linkage Levels Connect the map Initial Precursor Behavior to the items developed. Testlet IP Distal Precursor Proximal Precursor Behavior Behavior Testlet DP Testlet PP Target Successors Behavior Behavior Testlet T Testlet S

10 Fractions M.EE.3.NF1-3 Differentiate a fractional part from a whole 10

11

12 Assessment Design Testlets Usually assess 1 EE / linkage level 3-5 items Engagement activity ELA: Text or writing task Math: Context at beginning, carried throughout Teacher-administered or computeradministered (MC, sorting, matching)

13 Special user interface designed for DLM Customized presentation through Personal Needs & Preferences (PNP) profile

14 Sample Testlets ELA: RI 5.8: Identify the relationship between a specific point and supporting reasons in an informational text Target Level Node: Same as EE Math: 7.NS.2.a: Solve multiplication problems with products to 100 Initial Precursor Level Nodes: Recognize set, recognize separateness

15 Two Levels of Writing Assessments Conventional Writing Using orthography to communicate Selecting topics Generating and communicating facts, ideas and details in writing Emergent Writing Behaviors that precede conventional writing Communication about choices and information Cognitive processes that are precursors to conventional writing

16 Example Task/Rating in an Emergent Writing Testlet Screen 1 SAY: What do you want to write about? Choose one. WAIT AND OBSERVE: Give the student time to choose one of the objects.

17 Example Task/Rating in an Emergent Writing Testlet Screen 2 Choose the highest level that describes your observation. Makes a choice between objects. Attends to objects but does not make a choice. Seeks objects but does not make a choice. Attends to other stimuli. No response.

18 Example Task/Rating in an Conventional Writing Testlet Screen 1 SAY: What other facts, ideas or details can you write about [topic]? WAIT: Give the student time if needed to think about additional facts, details or other information about the topic. SAY: Write about them. WAIT AND OBSERVE: Give the student time to write facts, details or information about the topic.

19 Example Task/Rating in an Conventional Writing Testlet Screen 2 Choose the highest level that describes your observation. Independently wrote 3 or more facts, ideas or details related to the topic Independently wrote at least 2 facts, ideas or details related to the topic Wrote one fact related to the topic Wrote multiple facts, ideas or details unrelated to the topic. Wrote a single fact, idea or detail unrelated to the topic.

20 ACCESSIBILITY

21 Accessibility Accessible Content Personal Learning Profile Technology

22 Accessible Content Testlet levels Vocabulary Multiple and alternate pathways Special item writing guidelines Careful attention to unnecessary complexity

23 Personal Learning Profile Personal Needs and Preferences (PNP) Display Language & Braille Audio & Environment First Contact Communication Academics Sensory characteristics Motor characteristics Computer access

24 Technology Special user interface designed for DLM Dynamic routing

25 Accessibility Manual 1. Include Eligible Students in the DLM Assessment 2. Learn About the DLM Accessibility Features: What Does DLM Provide? 3. Discuss and Select Appropriate Supports and Tools: Considerations for IEP Teams 4. Enter Appropriate Supports into the DLM System 5. Prepare for the Assessment: Using the Chosen Accessibility Features 6. Evaluate the Accessibility Features Used After the Assessment

26 1. Supports activated by PNP A. Magnification B. Invert Color Choice C. Color Contrast D. Color Overlay E. Read Aloud with highlighting Text to Speech text only text & graphics graphics only nonvisual

27 2. Supports Requiring Additional Materials/Tools F. Uncontracted Braille G. Single-switch system/pnp enabled H. Two switch system I. Administration via ipad J. Adaptive equipment used by student K. Individualized Manipulatives

28 3. Supports Provided Outside the System L. Human Read Aloud M. Sign interpretation of text N. Language translation of text O. Test administrator enter responses for student P. Partner Assisted Scanning

29 Personal Needs & Preferences (PNP) Profile Display Enhancements Language & Braille Audio & Environment Other Supports

30 Students with Blindness/Visual Alternate forms Impairments Content would normally be represented on computer but translated into teacheradministered testlet Braille Alternate pathways

31 Braille Very small number of DLM students use Braille Forms will be available: Beginning spring 2015 operational window Only in upper linkage levels Delivery method still under evaluation by state partners

32 Hardware/Software No software costs (KITE is free) PNP features built in Use devices already available (interactive whiteboard, 1-switch or 2-switch systems, headphones, etc.) Option for test administrator to enter responses if there is a gap

33 HOW THE YEAR END MODEL WORKS

34 Year End Assessment Model Optional Instructionally- Embedded Optional Instructionally Embedded Testlets can be delivered for select Essential Elements System recommends linkage level; teacher can accept or override Student performance does not contribute for summative purposes Required Spring Testing Testlets cover the whole blueprint Dynamic routing supports adaptivity between testlets (system determines linkage levels) Scores are used for summative purposes

35 Content of the Assessment Grades 3-8 and high school English Language Arts (ELA) Mathematics Blueprints: Consortium approved a subset of Essential Elements Consortium set minimum requirements for breadth of coverage

36 Test Delivery Optional Instructionally Embedded One testlet per EE chosen in ITI Teacher chooses linkage level Each testlet assigned separately Test administrator schedules sessions within testing windows Spring (End of Year) 4-7 testlets per subject depending on grade Entire blueprint covered System chooses linkage levels Each testlet assigned separately Test administrator schedules sessions within window

37 Testlet Delivery System has testlets available at all 5 linkage levels for every part of the test Part 1 IP Part 2 IP Part 3 IP Part 4 IP Students take one testlet from one level for each group DP PP DP PP DP PP DP PP System determines which level to deliver to the student T S T S T S T S

38 Testlet Delivery Part 1 Part 2 Part 3 Part 4 IP IP IP IP DP DP DP DP PP PP PP PP T T T T S S S S

39 Testlet Delivery Part 1 Part 2 Part 3 Part 4 IP IP IP IP DP DP DP DP PP PP PP PP T T T T S S S S

40 Testlet Recommendations & Assignments

41

42 Performance Profile Part 1: Overall Results

43 Performance Profile Part 2: Conceptual Areas

44 Performance Profile Part 3: Narrative

45 THE KITE SYSTEM

46 Installed on testing devices Student login credentials

47 Website access (use Firefox) Staff login credentials

48 FROM THE TEACHER S PERSPECTIVE

49 Key Steps 1. Training & PD 2. Eligibility and accessibility decisions 3. Manage student data in Educator Portal 4. Select content; provide instruction 5. Optional: prepare for & deliver instructionally embedded assessments 6. Prepare for & deliver testlets during spring window 7. Plan for next year

50 Required Training vs. Professional Development Training Multiple modules Available in self-directed and facilitated formats (states/districts decide which to use) Covers critical content for managing and delivering the assessment Required for all test administrators No access to student logins without it PD Multiple modules Available in self-directed and facilitated formats (states/districts decide which to use) Covers a variety of topics to support instruction in academics States determine what is required/optional

51 Other Resources for Test Administrators Test Administration Manual With checklist Accessibility Manual Information about EEs and levels Access to released testlets and practice activities Educator resource page with supplemental info & videos

52 Management of Student Records Accuracy of roster Correct students, correct information First Contact PNP: Accessibility features discussed with IEP teams Documented in IEP Instructional plans (if using ITI)

53

54 Verify Students & Rosters

55 First Contact

56 PNP

57 Instructional Tools

58 DISTRICT SUPPORT FOR IMPLEMENTATION

59 DLM-Defined Roles Data Steward Technical Liaison Assessment Coordinator Test Administrator (teacher, educator) Individuals may fill more than one role.

60 Some States Have Other Roles Regional technical assistance provider Mentor/facilitator Vendor (other support services) Others? How do these fit with DLM roles?

61 Data Steward Responsibilities Gather, validate, and upload teacher, student, and roster data into Educator Portal Update records as needed

62 Technical Liaison Responsibilities KITE Client Troubleshooting Educator Portal Troubleshooting CSV files Whitelisting *.cete.us LCS, Local Caching Server

63 Assessment Coordinator Responsibilities Manage district/school staff roles Develop and implement training plan Develop and implement communication plan Monitor / support test preparations and administration

64 Checklist Categories: Before beginning assessment Information/resources Steps to take During (Optional) instructionally embedded assessment Spring assessment Prepare for next year

65 Before Assessment Manage district/school staff roles and responsibilities. Identify test administrators/examiners Identify a Technical Liaison Share roles/responsibilities and timelines with staff Share DLM policies and procedures

66 Before Assessment Prepare for assessment implementation Disseminate Technical Liaison contact information Encourage sign up for test notifications Establish communications between Data Steward, Technical Liaisons, and DLM. Develop communication channels with district personnel (updates, etc.) Monitor creation of Educator Portal user accounts Follow up with test administrators Establish a test security plan Troubleshoot issues

67 User Account Creation Must be on the list and active

68 Common Roles in Educator Portal District Test Coordinator Teacher Building Test Coordinator Building & District Users

69 Assigning Permissions in EP State District District Building

70 Unusual Cases Contract teachers Out of district students Charter schools

71 Before Assessment Develop and implement a comprehensive training plan OR implement your state s training plan for district staff and test administrators. Review the checklists (TL & TA) Encourage Technical Liaisons to complete training Deliver required test administration training Monitor educator completion of required training Identify additional training needs Monitor participation in DLM professional development

72 Required Training

73 Monitoring Completion of RT Now Participant can take screen shot of transcripts area State PD administrator can look up individuals State PD administrator can run PD completion report, share with districts Starting in January New District PD administrator role look up records, run reports

74 Before Assessment Monitor and support test preparations. Remind Technical Liaisons and Test Administrators about important deadlines. Monitor identification of students for DLM Ensure every student in the system is assigned to a teacher Monitor completion of Access (PNP) Profile and First Contact Assist test administrators with technology preparations Support test environment preparation Manage student usernames and passwords for KITE

75 Monitoring in EP

76 Before Assessment Develop a plan to facilitate communications with parents and district staff. Consider what parents and district staff will need and want to know about DLM assessments in Determine the frequency of communications to various audiences. Deliver communication to various audiences on the timeline in your communication plan. Relay new information from DLM and your state and district to appropriate audiences as needed.

77 Before the Spring Window Support Test Administrators in rechecking student enrollment, demographic information, PNP settings, and First Contact responses.

78 Checklist Categories: Before beginning assessment Information/resources Steps to take During instructionally embedded assessment Spring assessment Prepare for next year

79 Spring Window Monitor and support test administration. Facilitate communications between DLM staff and district staff during testing window(s). Obtain and deliver Braille forms as needed. Monitor test activity, including test completion prior to end of testing window. Serve as the first point-of-contact for teachers, answering questions about the assessment. Troubleshoot issues as they arise. Serve as the point-of-contact to the DLM help desk.

80 Monitoring Available in January: Report showing number of testlets completed and in progress vs. what is intended

81 Checklist Categories: Before beginning assessment Information/resources Steps to take During instructionally embedded assessment Spring assessment Prepare for next year

82 Resources & Next Steps Bookmark your state s page on the DLM website. Use the resources on your state s page. Activate your Educator Portal account by following the instructions in the KITE activation .* Complete the Security Agreement in Educator Portal profile.* Sign up for DLM test notifications during the year.

83 Website Resources

84 Help Desk Support Three tiers of support for local users Information published in the front of all manuals Toll-free phone number Extended hours during testing windows with ticketing system Answers customized per testing model & for unique state policies

85 Communication KITE system Status on website Testing updates subscription Special to states Other issues Special to states Additional postings, announcements as warranted

86 OTHER DISCUSSION TOPICS

87 What questions do you have?

88 THANK YOU! For more information, please contact: or Go to: For Professional Development, contact: The present publication was developed under grant X from the U.S. Department of Education, Office of Special Education Programs. The views expressed herein are solely those of the author(s), and no official endorsement by the U.S. Department should be inferred.