Preparing for 2015 Smarter Balanced Summative Assessments

Transcription

1 Preparing for 2015 Smarter Balanced Summative Assessments Rick Phelan February 24, 2015 Sonoma County Office of Education Appreciations to Rocklin Unified School District and the California Department of Education for slide deck elements.

2 Goals Participants will: o Understand roles & responsibilities for administering the 2015 Smarter Balanced Summative Assessments o Obtain tools to support scheduling and logistical planning o Understand student access options o Gain materials to prepare test administrators/proctors o Learn about communication resources for students & parents o See an overview of the Test Operations Management System 2

3 Pre- Test 3

4 SBAC Test Design 4

5 Smarter Balanced Assessment System Summative assessments benchmarked to college and career readiness Common Core State Standards specify K 12 expectations for college and career readiness Teachers and schools have information and tools they need to improve teaching and learning All students leave high school ready for college and career Formative assessment tools and practices for teachers to improve instruction Interim assessments flexible, open, used for actionable feedback

6 Smarter Balanced Computer Tests Test Administrators Directions for Administration (DFA) are located at: Grades 3-8 and 11 in ELA and Math The only students exempt from participating in the test are: Students participating in the Alternate Assessment English Language learners who have been attending schools in the United States for less than 12 months at the time of the test Computer-adaptive test (CAT) in each content area Performance Task (PT) in each content area Classroom activity to familiarize students with question content Students respond to the PT through online platform 6

7 Smarter Balanced Item Response Types Multiple choice one correct response multiple correct responses two-part Matching tables yes/no true/false Fill-in tables Select or order text or graphics Drag and drop Graphing Equation or numeric response Short text Long essay

8 SBAC Practice Tests Allow students and administrators to become familiar with and experience Grade-level assessments A variety of grade-specific items and difficulty levels Performance Tasks Format and structure of the operational assessments Allow students to become familiar with accessing the test (log-in process), testing software and navigational tools. 8

9 Interim Assessments The Smarter Balanced Interim Assessments are comprised of interim comprehensive assessments (ICAs) and interim assessment blocks (IABs) ICAs - Examples of the use include: Beginning of the year a student from another state is given the previous year s ICA. Mid-year a teacher gives an ICA to gauge how students might perform on the summative assessment. Scoring Protocol for Interim Assessments will be available at the end of February. 9

10 Interim Assessments (cont.) The Smarter Balanced Interim Assessments are comprised of interim comprehensive assessments (ICAs) and interim assessment blocks (IABs) IABs - Assess fewer sets of skills, and: Use the same targets, by grade level, as the summative blueprints. Consists of short, focused sets of items. Provide information about a student s strengths and needs in relation to the assessment targets. Offer varied blocks by grade level and subject area. 10

11 SBAC Roles & Responsibilities 11

12 Roles and Responsibilities District Test Coordinator (DC) Designate Site Coordinators (SCs), add them to the Test Operations Management System (TOMS) Promote understanding of SBAC with stakeholder groups Coordinate with SC s so that tests are administered in the appropriate grade(s) and content area(s) Set appropriate supports and accommodations in TOMS Ensure that the SC s and TA s are appropriately trained regarding test administration and security policies Report test security incidents to CDE Work closely with Computer Lab Tech/Site Tech Coordinators Relay issues/questions to the CalTAC call center 12

13 Roles and Responsibilities School Test or Site Coordinators (SC) Collect security affidavits from Test Administrators and retain at test site Train TA s and verify their accounts on TOMS Promote understanding of SBAC with stakeholder groups Verify supports and accommodations in TOMS minimum of three days before your testing window Print student log-in strips/cards (confirmation code and SSID) Confirmation code is the student s first name from SIS If you have a new students, notify your district CALPADS engineer Ensure that testing at the school sites is conducted in accordance to test security policies Create/approve test schedules and procedures for the school. Monitor testing and address problems, as needed Report testing security incidents to the DC 13

14 Roles and Responsibilities Test Administrator (TA) Completely read and sign security affidavit Promote understanding of SBAC with stakeholder groups Complete test administration training Log in and verify access to the testing system using the CAASPP account sent via View student information prior to testing, to ensure it is correct Ensure that appropriate designated supports and accommodations have been set prior to test Set up and proctor testing Report all potential test security to the SC and District CAASPP Coordinator 14

15 Roles and Responsibilities Technical Support Staff Read and sign security affidavit Complete test administration training Ensure all student computers can launch and run secure browser application Ensure bandwidth test is completed at each school prior to testing windows Ensure all student computers have functional headphones Assist the TA with set up and test proctoring Increase comfort levels and reduce anxiety of students & staff 15

16 SBAC Technology 16

17 Technology Setup Insure devices are operational Secure browsers properly installed a minimum 2 weeks before testing Necessary hardware available and working (headphones, external keyboards, etc.) Content filters, firewalls, and proxy servers configured Stable (wired or wireless) internet connection 17

18 Secure Browser The Secure Browser is: Specifically designed for CA to provide secure access Prevents students from accessing other hardware and software functions NOTE: Test Administrator computers will use a standard Web browser 18

19 Secure Browser Students must use the browser to log on to take a summative test Prior to testing, TA s should verify that: External user applications are closed (browser will not work if they re on) Testing does not occur on computers with dual monitors 19

20 Technology Tools & Resources 20

21 SBAC Scheduling Issues 21

22 Smarter Balanced Computer Based Tests Time Estimates 22

23 Smarter Balanced Test Schedule Overall testing will take approximately 7 to 8.5 hours Recommended: Non-PT first, PT second (Performance Task) Classroom activity should be completed within 3 days prior to starting PT ELA PT s have 2 parts and can be completed in 2 days Math PT s have 1 part and can be completed in 1 day Recommended: PT should not be administered on the same day as the non-pt 23

24 Smarter Balanced Test Schedule (continued) SC s must ensure that TA s know which students take which tests Schedule activities for students not taking exam or for those who finish early (eg. may read a hardcopy book A students CAT test remains active until the student completes and submits the test or 45 calendar days after the student has begun the CAT A student s PT remains active for 10 calendar days after the student has begun. 24

25 2015 SBAC Scheduling Estimator hfp:// estimator xls 25

26 SBAC System Operations 26

27 Three Systems Connect to Deliver Tests to Students CALPADS Student enrollment information TOMS Student demographics Manage test users Manage test windows Manage student supports Student rosters Smarter Balanced Verify student test settings Select student tests Test delivery portal 27

28 CALPADS Sole source for student demographics and program participation data for TOMS Districts cannot update student demographics or program participation data in TOMS; must be updated in CALPADS first Contact your CALPADS administrator to correct student grade level or demographics Data from CALPADS is rolled to TOMS daily and the turnaround time is about 48 hours. If a new student enrolls during the District Testing window contact your CALPADS administrator to get them set up in TOMS 28

29 Test Operations Management System (TOMS) TOMS is a password-protected Web site that provides administrators with tools to manage users and students participating in Smarter Balanced assessments Repository for student information, including initial test settings and accommodations Different levels of user access (DC, SC, TA) District and school staff must set designated supports and accommodations for students in TOMS prior to testing In TOMS, DCs and SCs can Add and manage users View student roster and update student test settings 29

30 Accessing TOMS Step 1. hfp:// californiatac.org/ Step 2. Log In Step 3. Navigate the System District and site coordinators authorize users for the system. Authorized users receive an with login instructions and must activate their login before the temporary password expires. Usernames and passwords used to access last year s TIDES system are no longer active. A manual on how to use TOMS is available: 30

31 Operating an SBAC Test Session STEP 1: Preparation Prepare computers o Student computers (No testing on computers with dual monitors ) A secure browser must be installed on all student computers/devices (updated o annually) Test Administrator computers Use a standard Web browser. Prior to testing, TAs should verify on student computers that external user applications are closed to allow the lockdown browser to work Secure browser available for download through the portal: Prepare student login information o SSID, and confirmation code (CALPADS student first name) for login. This may be provided on a card or piece of paper. NOTE: Student personal information is secure and must be collected and securely destroyed at the end of a test session. Verify student test settings Ensure that the correct student test settings have been uploaded to TOMS (see the later section on accommodations and designate supports) 31

32 Smarter Balanced- Additional Features Test Settings and Accommodations Before you approve students to test, you can review student test settings. Only DC and SC may change certain settings If changes are made, select [Set] to confirm the current test settings and return to the list of students awaiting approval. You will still have to approve the student for testing. Select [Set & Approve] to establish the updated settings and approve the student for testing. Select [Cancel] if you want to return to the Approvals and Student Test Settings screen. Note: Translations (stacked) (for math items) and turning off universal tools only occurs in the test delivery system. 32

33 Operating an SBAC Test Session STEP 2: TAs Log into the system and generate a test session o Go to and select Test Administrator Interface. o Enter your user name and password. o Select the test(s) you will be administering during that session and click on Start Session. o Write the test Session ID on the board or large paper where students can see it. Keep a record of this test session ID in the event that the session suddenly shuts down. o Do NOT navigate away from the TA interface page. The session will stop and all students will be logged out. o Read the testing script from the Test Administrator Manual o Provide students with their SSID, and confirmation code (CALPADS student first name) for login. o Make sure students know that the Test Session ID must be entered exactly as it is written, without extra spaces or characters. 33

34 Operating an SBAC Test Session o Step 3: Students log into the system To log in to the Smarter Balanced test, each student will need: Confirmation Code: Student s legal first name as spelled in CALPADS State-SSID: State abbreviation (CA) followed by a hyphen and the student s Statewide Student Identifier Session ID Students will select their test Students must wait for TA approval for entry into the test session Students must then verify their test 34

35 Operating an SBAC Test Session Step 4: Approving Students for Testing TAs can either select [Approve] for each individual student (recommended) or select [Approve All Students]. Reminder: If any student s test settings are incorrect, do NOT approve that student. To refresh the list of students awaiting approval at any time, select the [Refresh] button at the top of the pop-up window. Do not use the browser refresh button. 35

36 Operating an SBAC Test Session Step 5: Students Begin their Test Students will be taken through a sound check verification (two if text to speech is activated) Followed by a Test Instructions and Help screen 36

37 Operating an SBAC Test Session STEP 6. Monitoring Student Progress The table displays: Students who have logged in and been approved for testing. Test completion Status Print requests Paused tests Additional tests can be added to a session; you will receive a confirmation message box 37

38 Ending an SBAC Test Session STEP 7: Ending the Test Students complete the test: TA ends the test session: To stop the ENTIRE session (and pause tests for all students in the session): Select the [Stop Session] bufon in the upper- left corner of the screen. An Important! box will appear, requesting verification to end the session and log students out. Select [OK] to continue or [Cancel] to keep the test session open. Users should exit or log out of the Test Administrator Interface only after stopping the test session. 38

39 Pausing and Inactivity Rules Through the TA Interface, TAs may: Stop an entire session; or Pause individual student tests. Do NOT stop tests without forewarning students Users should exit or log out of the Test Administrator Interface only after stopping the test session. If there is a technical issue (i.e. power outage or network failure), students will be logged out and the test will automatically be paused. Regardless of when or how users log out or navigate away from the Test Administrator Interface, student data will NOT be lost. Note: If a test is paused for any reason or length of time, the student must log in again to resume testing. Highlighted text and item notes will not be preserved on the computer adaptive tests. There is no pause option for the PT (performance task). Note for ELA (2 parts) a student cannot go pack to part 1 after completing part 2. 39

40 SBAC Pausing and Inactivity Rules As a security measure, students are automatically logged out of the test after 20 minutes of inactivity. When this occurs, students are: Presented with the test page containing the test item he or she was last working on (if page contains at least 1 unanswered item) OR Presented with the next test page (if all items on the previous test page were answered); NOT permitted to review or change any test items on previous test pages. Activity means: Selecting an answer Using a navigation option in the test (i.e. selecting [Next] or [Back], using the Past/Marked Questions drop-down list) NOTE: Moving the mouse or selecting an empty space on the screen is NOT considered activity. Before the system logs out, a warning message will be displayed. If a student does NOT select [Ok] within 30 seconds, the student will be logged out. Selecting [Ok] will restart the 20 minute inactivity timer. 40

41 General Rules & Conditions for SBAC Student Responses Students must answer all test items on a page before moving on to the next page Students may review already answered questions before submitting test Students may not return to a test segment once it has been completed and submitted Students must answer all test items before the test can be submitted Students may not return to a test once it has been completed and submitted There is a difference between a segment and a session Expiration and pause rules vary between PT and non-pt 41

42 Administration of the Classroom Activity and Performance Task

43 Classroom Activity Administration Guidelines Classroom Activity Purpose: To provide students with important context, concepts, and key terms prior to taking the PT. For each content area, all students will have a classroom activity before they complete the PT. May be administered in a classroom or any other appropriate space. May take place on a separate day from the PT. Recommended: No more than a 3-day lapse between the classroom activity and the PT administration. TA may need a chalkboard or dry-erase board. 43

44 Classroom Activity Administration Guidelines (cont.) Computers, projectors, and other technology are allowed but not required for the classroom activity. Involves the participation of all students in an instructional task. PT will be completed by individual students in the computer-based test delivery system. Students may take notes during the classroom activity, but the notes must be collected before proceeding to the PT. 44

45 Classroom Activity Administration Guidelines (cont.) Designed to fit into a 30-minute window. Will vary due to complexity of topic and individual student needs. The SC should download the assigned classroom activity and provide to TAs at least 1 2 days prior. Download from After receiving the activity from the SC, the TA or classroom teacher should administer the classroom activity. Provide the appropriate accommodations for students who normally use them during instruction. 45

46 Classroom Activity and Performance Task Administration Ø Step 1: Identify the classroom activities for your school. Ø Step 2: Download the classroom activity. Ø Step 3: Review the teacher directions for the classroom activity carefully. Ø Step 4: Complete the classroom activity. Ø Step 5: Administer the PT to students online. The students must work independently 46

47 Universal Supports, Designated Supports, & Documented Accommodations

48 Accessibility and Accommodations Three categories of support Universal Tools: o Available to all; based on student preference and selection (e.g., breaks, strikethrough, highlighter). Designated Supports: o Need to be indicated by an educator or group of educators (e.g., color contrast, read aloud, glossary, magnification). Accommodation: o Based on individualized education programs (IEPs) or Section 504 plans (e.g., American Sign Language, scribe). 48

49 Accessibility and Accommodations Two systems of delivery Ø Embedded supports: digitally delivered through the technology platform(i.e., digital notepad, calculator) MUST be activated in TOMS prior to testing. Ø Non-embedded supports: provided by the District(i.e., adapted keyboard, multiplication table) but are still noted in TOMS 49

50 Accessibility and Accommodations 50

51 Accessibility and Accommodations Questions: How do I know what supports are available? How do I match the IEP and 504 supports to the supports listed? Answers and Resources: Matrix One: Universal tools, designated supports, and accommodations for the CAASPP system Shows tables of all available items by exam and level of support Smarter Balanced Usability, Accessibility, and Accommodation Guidelines Provides details about each of the supports and contexts in which they are appropriate Describes who is eligible for them and guides the decision making process Detailed Q & A section in an appendix ISAAP Tool & SEIS Helps staff select and organize appropriate support categories Summarizes students supports that can be uploaded to TOMS 51

52 Accessibility and Accommodations Not all tools, supports, and accommodations are available for all exams or to all grade levels. For example: o ELA a dictionary is not allowed on short paragraph responses but are allowed on the PT long essay. o ELA - read-aloud of passages or text-to-speech is not allowed on passages for students in grades 3-5. o Math multiplication tables are not allowed in grade 3. When an IEP or 504 specifies that a disallowed tool is required for a student to participate in testing, special permission must be granted to use these Individualized Aids. o o The District CAASPP Coordinator or the CAASPP Test Site Coordinator must submit the CAASPP Individualized Aid Request Form. Approval to use these is granted by the California Department of Education. When an Individualized Aid changes the construct being measured, the scores for that student will not count towards the participation rates. This is similar to the modifications policy under the prior STAR program the student gets a score report, but the score is not used for accountability. 52

53 Accessibility and Accommodations Planning Ahead Determination of which accommodations an individual student will have available for the assessment is necessary because these accommodations must be made available before the assessment, either by entering information into the TOMS for embedded accommodations, or by ensuring that the materials or setting are available for the assessment for nonembedded accommodations. Accommodations could increase cognitive load or create other challenges for students who do not need them or who have not had experience using them. It is wise for students to practice the tests using the selected tools. Parents/guardians should be made aware of the conditions under which their child participates in the assessment. Once the appropriate supports are identified, site and district coordinators will work together to ensure that they are uploaded to TOMS. TA s are not authorized to change student test settings in TOMS Prior to testing review the students test settings and verify that they are correct. 53

54 Accessibility and Accommodations Online Resources Smarter Balanced Usability, Accessibility, and Accommodations Guidelines: SmarterBalanced_Guidelines.pdf Smarter Balanced Usability, Accessibility, and Accommodations Implementation Guide: Usability-Accessibility-and-Accommodations-Implementation-Guide.pdf Guidelines: Frequently Asked Questions: SmarterBalanced_Guidelines_FAQ.pdf Accessibility and Accommodations for CAASPP : An Overview archived Webcast Matrix One: Using the CAASPP Individual Student Assessment Accessibility Profile (ISAAP) Tool video tutorial: Embedded Universal Tools and Online Features training module: 54

55 Test Security 55

56 Test Security There are 3 different types of test security incidents Impropriety (Correctable and containable at local level) o o o Unusual circumstance, low risk of affecting student performance, test security, or test validity Must be reported to District CAASPP Coordinator or Test Site Coordinator Example: Students talking during testing Irregularity (Correctable and containable at local level) o o o Unusual circumstance, may affect student performance, test security, or test validity Must be reported to District CAASPP Coordinator within 24 hours Examples: Student cheating or providing answers to another; student accessing or using electronic equipment (e.g., cellphone) Breach (External implications for the Consortium) o o o Event that threatens test validity (e.g., release of secure materials) A breach incident must be reported to District CAASPP Coordinator within 24 hours Examples: Administrator or coordinator modifying student responses or record at any time Administrator allowing students to take home test items, passage, prompts, or scratch paper 56

57 Resources Sonoma County Office of Education SBAC Support Pages Manuals o Administration Manuals o Universal Tools, Designated Supports, and Accommodations o TOMS Users Guide o Administration of Paper and Pencil Tests Smarter Balanced Test login site 57