C A L I F ORN I A. Assessment of Student Performance and Progress

Transcription

1 C A L I F ORN I A Assessment of Student Performance and Progress Smarter Balanced Interim Assessment Reporting System User Guide for California Winter/Spring 2015 Smarter Balanced Interim Assessments for English Language Arts/Literacy and Mathematics Updated April 21, 2015 Prepared by the Smarter Balanced Assessment Consortium California customization prepared by Educational Testing Service

2 CONTENTS IA Reporting System User Guide 1.0 Overview Background Audience for this Manual Interim Assessment Types and Reports Security and Privacy Accessing the Interim ASSESSMENT Reporting System User Roles in the Reporting System Prior to Accessing the IA Reporting System Accessing the IA Reporting System Logging Out of the IA Reporting System Report Descriptions Individual Student Report List of Students in a Grade Report Data Downloads Student Assessment Results Printable Student Reports Reporting Features Summary Definition of Terms Accessibility Features of the Reports Accommodations Achievement Levels and Proficiency Claims Reporting Context Security, Access Patterns, and Single Sign-On (SSO) Error Bands and Standard Error of Measurement Interim Assessment Blocks (IAB) Reporting Interim Comprehensive Assessments (ICA) Reporting Assumptions Help HTTPS Pickup Zone Language Support Legends Logging Navigation Permissions Printing, Individual and Batch Security Scale Score Section 508 Compliance Technical Requirements Documented Assumptions and Clarifications Appendix A. Registering Users and Assigning Permissions for the Interim Assessment Reporting System About Managing Users and Their Roles Adding the Reporting Roles for an Existing User Adding New Users Registering Institutions (Schools) Change Log LIST OF FIGURES Figure 1. Individual Student Report Figure 2. Individual Student Report Information Indicators, Assessment Selector Figure 3. Breadcrumb Navigation Figure 4. Individual Student Report Legend Figure 5. Individual Student Report Overall Scores of 61- Updated April 21, 2015

3 Figure 6. Individual Student Report Overall Score Bar Figure 7. Individual Student Report Mathematics Claim Levels Figure 8. Individual Student Report ELA/literacy Claim Levels Figure 9. Individual Student Report Accommodations Reporting Figure 10. List of Students in a Grade Report Figure 11. Assessment Selector for a List of Students in a Grade Figure 12. Previous Year Warning Figure 13. Mathematics Overall and Claim Scores Figure 14. Legend for List of Students in a Grade Figure 15. Download Options Window Figure 16. Sorting for List of Students in a Grade Figure 17. Narrowing the List of Students in a Grade Figure 18. Download Popup Window Figure 19. Download Printable Student Report Options Figure 20. Download Confirmation Window Figure 21. Download Printable Student Report Options Figure 22. Cover Page for Printable Student Reports Figure 23. Segments Cover Page for Printable Student Reports Figure 24. Accommodations Display Figure 25. Claims Score Illustration Figure 26. Error Band Illustration Figure 27. Reporting Landing Page (for logged out users) Figure 28. Breadcrumb Navigation Figure 29. Download Popup Window Figure 30. Printable Student Report Options Figure 31. Download Confirmation Window Figure 32. List of Students in a Grade, Download Current View as CSV example LIST OF TABLES Table 1. ICAs and IABs available per student grade level for the school year... 5 Table 2. Roles in the IA Reporting System... 8 Table 3. Reporting Features Summary Table 4. Crosswalk of the Interim Assessment Achievement Levels to the Summative Assessment Achievement Levels Table 5. Claims List Table 6. Claim Level Descriptors Table 7. ELA/literacy Reporting Achievement Level Descriptors Table 8. Mathematics Reporting Achievement Level Descriptors of 61- Updated April 21, 2015

4 1.0 OVERVIEW IA Reporting System User Guide This guide provides detailed instructions for educators to use the Smarter Balanced Interim Assessments (IA) Reporting System. Please note: California local educational agencies (LEAs) will use a different system for accessing results from the Smarter Balanced Summative Assessments. 1.1 Background The Smarter Balanced Interim Assessments are provided to LEAs by the state as part of the California Assessment of Student Performance and Progress (CAASPP). Students take the interim assessments online, and test administrators (TAs) or other authorized LEA or school personnel score student responses that must be hand scored. To complete the hand scoring of these items, TAs use the Interim Assessment Hand Scoring System (IAHSS) provided by the state testing contractor, Educational Testing Service (ETS). Student results for interim assessments that have complete scored responses are sent to the IA Reporting System operated by Smarter Balanced. Both the IAHSS and the IA Reporting System are available through the CAASPP Portal at Audience for this Manual This manual provides guidance to LEA CAASPP coordinators and other authorized users of the IA Reporting System. 1.3 Interim Assessment Types and Reports Interim assessments provide educators with the information necessary to guide instruction that will benefit each student in California. Two types of interim assessments are available through the Smarter Balanced Interim Assessment System: interim comprehensive assessments (ICAs) and interim assessment blocks (IABs). ICAs are built on the same blueprints as the Smarter Balanced Summative Assessments. The ICAs include the same item types and formats, including performance tasks, as the summative assessments, and yield results on the same vertical scale. IABs focus on smaller sets of targets and therefore provide more detailed information for instructional purposes. The IABs yield overall information for each block. Interim assessments have been built for grades 3 through 8 and grade 11. LEAs have the flexibility of using the Smarter Balanced Interim Assessments with students in other grade levels to gather information on what students know and are able to do with regard to the English language arts/literacy and mathematics content standards. Table 1 specifies the ICAs and IABs that are available per student grade level for the school year. Please note that the grade-level eligibility reflected in the table pertains only to the school year and may change in the future years. Interim assessment results are available: Within hours after a student completes an IAB with no local hand scoring requirement. Within hours after any required local hand scoring is completed. -4 of 61- Updated April 21, 2015

5 Table 1. ICAs and IABs available per student grade level for the school year Student Grade Level K 2 Grade 3 3 Grade 3 Grade 4 4 Grade 3 Grade 4 Grade 5 5 Grade 4 Grade 5 Grade 6 6 Grade 5 Grade 6 Grade 7 7 Grade 6 Grade 7 Grade 8 8 Grade 7 Grade 8 High School 9 Grade 8 High School 10 Grade 8 High School 11 Grade 8 High School 12 Grade 8 High School ICAs and IABs for the School Year* *Each ICA and IAB may be accessed by a student no more than two times for the school year. -5 of 61- Updated April 21, 2015

6 2.0 SECURITY AND PRIVACY IA Reporting System User Guide Interim assessments provide educators with the information necessary to guide instruction that will benefit each student in California. The Smarter Balanced Interim Assessment Reporting System (IA Reporting System) will be collecting and maintaining educational records that contain personally identifiable information (PII) on students. Smarter Balanced security controls are implemented to protect the information processed and stored within the system. Specifically, these control settings are designed to: Protect the privacy and confidentiality of the system information. Ensure that only authorized users (educators) access the system. Ensure that educators are uniquely identified when using the system. Associate and connect certain actions performed within the system to specific educators. Ensure that educators have access to perform only the actions they have been authorized to carry out. Considerations in Assigning Access to the IA Reporting System IA Reporting System users are an integral part in ensuring that the security controls provide the intended level of protection. All LEA representatives who are authorized to access the IA Reporting System are required to protect interim assessment reporting information in any form. This includes information contained on printed reports, data downloaded onto computers and computer media (e.g., diskettes, tapes, compact disks, thumb/flash drives), user computer monitors, or any other format. The California Department of Education (CDE) will provide IA Reporting System access to the CAASPP coordinator of each California LEA. Each LEA CAASPP coordinator, in turn, is authorized to grant access to district and school staff. Prior to granting access to additional users, the LEA CAASPP coordinator should consider state and federal privacy laws as well as local district policies. The CDE does not provide legal interpretation of state or federal privacy laws. LEAs should consult with their local legal offices to ensure that local processes are in compliance with applicable state and federal laws and regulations. -6 of 61- Updated April 21, 2015

7 3.0 ACCESSING THE INTERIM ASSESSMENT REPORTING SYSTEM PII is protected through the latest encryption and security technology, and educators are only permitted to view data for students they have been explicitly authorized to view. Follow state and federal laws and district policy and only provide access to authorized individuals. IMPORTANT: Access to student PII is strictly regulated by state and federal laws and district policies. LEA CAASPP coordinators have been designated by their Superintendent at the beginning of the school year as authorized personnel. As authorized personnel, LEA CAASPP coordinators whose LEAs have administered interim assessments will automatically be provided access to the IA Reporting System by the CDE. Once given access to the IA Reporting System, the LEA CAASPP coordinator is able to provide access for other authorized LEA personnel. Before granting access to the IA Reporting System, LEA CAASPP coordinators should read and understand the user roles that are available and the implications of assigning the roles to authorized educators. Instructions for adding educators to the Smarter Balanced Administration and Registration Tool (ART) is included in Appendix A. 3.1 User Roles in the Reporting System The CDE will assign LEA CAASPP coordinators with three user roles: GENERAL, PII, and SAREXTRACTS. LEA CAASPP coordinators will be able to add authorized LEA personnel and assign them the appropriate reporting user roles. To access the IA Reporting System and see student data, authorized users must have the GENERAL and PII roles. Additionally, users may have the SAREXTRACTS role which allows users to download interim assessment test results. Table 2 describes each reporting role and the access provided by that role. -7 of 61- Updated April 21, 2015

8 Table 2. Roles in the IA Reporting System Role Display Name in ART GENERAL (with Responsible Entity Type as DISTRICT ) GENERAL (with Responsible Entity Type as INSTITUTION ) PII SAREXTRACTS Role Description This role is required for access to the IA Reporting System. This role is required for access to the IA Reporting System. This role provides access to individual student assessment results for all institutions under the associated entities. To activate this role, GENERAL must be assigned as well. This role provides access to download the reported data in detail for all students covered by the institution. To activate this role, GENERAL and PII must be assigned as well. IA Reporting System User Guide Role Example Any user who should have access to view student reports for an LEA (referred to as district in ART), schools in a district, and grades in a school. All authorized LEA personnel must be assigned this user role. Any user who should have access to view student reports for the assigned school, and grades within the school. All authorized school site personnel must be assigned this user role. Any user who should be able to see individual student assessment results. Interim results are only available at a student level, therefore, the LEA CAASPP coordinator must assign the PII role to any additional users. LEA CAASPP coordinators and other LEA staff and educators designated by the LEA CAASPP coordinator with authorization to view student test results through the IA Reporting System should be assigned this role. Note that users with this role may also print an individual student s test results. Any user who should be able to download the assessment results data that drives individual reporting. Only LEA personnel authorized to download the student test results should be assigned this user role as the data file includes all student test results for the institution. Please note that the only combinations of roles possible are GENERAL and PII and GENERAL, PII, and SAREXTRACTS. The GENERAL and PII combination will grant the user access to individual student assessment results for all institutions under the associated entities. The GENERAL, PII, and SAREXTRACTS combination will allow the user to access AND download the reported data in detail for all students covered by the institution. How to determine the scope of access (i.e., LEA level vs. school site level) is covered in Step 8 under the Adding the Reporting Roles for Existing Users section in Appendix A. -8 of 61- Updated April 21, 2015

9 3.2 Prior to Accessing the IA Reporting System IA Reporting System User Guide Before you access the IA Reporting System, check to see if the following actions have been completed: If you are the LEA CAASPP coordinator or site coordinator, Review the CAASPP Test Security Agreement for LEA CAASPP Coordinators and CAASPP Test Site Coordinators that should have been submitted at the beginning of the school year or when the coordinator was designated. Verify that you have received your login credentials from Smarter Balanced to the IA Reporting System. LEA CAASPP coordinators should contact the California Technical Assistance (CalTAC) Digital Library help Desk by phone at or by at for assistance with log in credentials. Provide access to other authorized LEA representatives including educators in your LEA or school. If you are an educator, Read and agree to the CAASPP Test Security Affidavit for Test Examiners, Proctors, Scribes, and Any Other Persons Having Access to CAASPP Tests (PDF) for accessing the student test results. Verify that you have received your login credentials from Smarter Balanced to the IA Reporting System. Contact your site or LEA CAASPP coordinator for assistance. Ensure that hand scoring has been completed for interim assessment questions that need to be hand scored. If the interim assessment that was administered included test questions that need to be hand scored, test results will not be generated until hand scoring has been completed. All ICAs and some IABs have test questions that need to be hand scored. Please refer to the Smarter Balanced IABs: Hand Scoring Requirements table on the CDE Web site. Test results for IABs that do not have hand-scorable items will be available in the IA Reporting System when the tests have been completed. Hand scoring of the interim assessments is a local decision and responsibility. -9 of 61- Updated April 21, 2015

10 3.3 Accessing the IA Reporting System IA Reporting System User Guide Follow the steps below to access and begin using the IA Reporting System. Users are only permitted to view data for schools they have been explicitly authorized to view. Step Action 1 Go to the CAASPP Portal ( and select the [Interim Assessment Reporting System] button. This will open the Smarter Balanced Reporting System landing page (shown below). -10 of 61- Updated April 21, 2015

11 Step Action 2 On the landing page, select the [Log In] Button. This will open the log in screen (shown below). 3 On the log in screen, enter your username and password this will be the same information you use to access the Smarter Balanced Digital Library. After you are logged on, this will open a screen that lists all LEAs with interim assessments ready for reporting. Districts that do not yet have interim assessment data ready for reporting will not be displayed on this list. 4 Search for your school district by either scrolling or entering your district name the Find box. -11 of 61- Updated April 21, 2015

12 Step Action 5 When you locate your district, click one time on the name of the district to display the schools with available interim assessment results. The words Interim Data Only will be displayed in the content area column (i.e., Mathematics, ELA/Literacy) for which interim assessment results are available. 6 Search for your school by either scrolling or entering your school name the Find box. Select your school by clicking one time on the school name. This will open a screen that displays the grade levels of the interim assessments for which reports are available, as shown below. -12 of 61- Updated April 21, 2015

13 Step Action 7 Select the grade by clicking one time on the grade. From here, you can experiment with features of the IA Reporting System. 3.4 Logging Out of the IA Reporting System The IA Reporting System will automatically log you out after a period of inactivity. However, because the system provides access to PII, it is important that you log out every time you have completed using the IA Reporting System. Follow the steps below to log out. Step 1 Action In the upper right corner of the IA Reporting System screen, select menu. to view the dropdown -13 of 61- Updated April 21, 2015

14 Step Action 2 Select the [Log Out] button. 3 You will be logged out of the IA Reporting System. Your browser will be directed back to the Smarter Balanced Single Sign On landing page. Close your browser as you would normally. -14 of 61- Updated April 21, 2015

15 4.0 REPORT DESCRIPTIONS IA Reporting System User Guide The IA Reporting System is an interactive, online reporting platform that provides reports and data from the Smarter Balanced Interim Assessments. The IA Reporting System does not create aggregate data reports for interim assessments. It does, however, enable individuals with the appropriate levels of access the ability to examine data at multiple levels and can create custom subsets filtered by gender or student demographic data (Limited English Proficient, Race/Ethnicity, Individualized Education Plan, Gender, 504, Economic Disadvantage, or Migrant Status). Interim assessment student reports can be downloaded as PDF files for easy printing and distribution to parents. This document provides a static view of the reports, with annotations, previously approved by Smarter Balanced, provided to explain the different features available. 4.1 Individual Student Report The individual student report presents individual student assessment results and illustrates performance against standards for a given ICA or IAB. The scores and descriptions provide context for understanding what the assessment has measured and how to interpret the scores and scoring categories. Educators, students, parents, and guardians may use this report to understand a student s achievement, progress toward mastery of the Common Core State Standards, and attainment of the academic knowledge and skills required to be college content- and career-ready. The report may provide context for a parent-teacher conference or, together with other instructional data, may help to identify areas for instructional focus. Figure 1 shows an example of the Individual Student Report. -15 of 61- Updated April 21, 2015

16 Figure 1. Individual Student Report Report Features Breadcrumbs: At the top of the page, users can navigate back to the grade, school, district, and state level reports. Please note: The grade level displayed on the report is the grade level of the interim assessment and will not necessarily correspond with the grade level in which the student is enrolled. Report Information: Detailed report information regarding the report s purpose and potential uses is accessible by pausing on or clicking the information icon next to the report s title. Assessment Selector: A dropdown list of all assessments that the student has taken, organized by assessment type and date administered. -16 of 61- Updated April 21, 2015

17 Top-Level Summary: The student s overall ICA mathematics and ELA/literacy scores and achievement level for the selected assessment. Print: The Print icon presents the option of printing a PDF version of the report. Users can select various print settings, including color and grayscale (for black-and-white printing). Achievement Level Descriptor: The description of the student s achievement level. Overall Score and Error Bands: The overall score for an ICA is indicated by a single number. This number is located visually along a continuum of scores. The score is surrounded by an error band, marked with brackets. This range indicates the range of scores that the student would be likely to achieve even if the student took the test multiple times. Claim Information: Mathematics and ELA/literacy are composed of claims. Within each claim, student performance is reported as Below Standard, At/Near Standard, or Above Standard. A description of each claim is provided. Accommodations: A list of accommodations available to and used by the student Detailed Descriptions Navigation A user with appropriate permissions may access an individual student report by clicking a student s name in the List of Students in a Grade report. Please note: The grade level displayed on the report is the grade level of the interim assessment and will not necessarily correspond with the grade level in which the student is enrolled. The student report displays results for one student on one assessment at a time. A user may navigate to other reports for this student via the Assessment Selector (see Figure 2). A user may use the breadcrumbs (Figure 3) to navigate up to the student s grade, school, district, or state. Figure 2. Individual Student Report Information Indicators, Assessment Selector -17 of 61- Updated April 21, 2015

18 Figure 3. Breadcrumb Navigation Informational Popup Windows Report Information, Interim Information, and Legend are all available by pausing on or clicking the respective icons. The Report Information popup contains additional information about the purposes and proposed uses of the report. For the Individual Student Report, a sample of the text follows: Purpose: This report presents individual student assessment results and provides guidance on how to interpret the results. Uses: Educators, parents, or students may use this report to understand student achievement, progress toward mastery of the Common Core State Standards and attainment of the academic knowledge and skills required to be college content- and career-ready. Understanding your score: The Smarter Balanced assessments may be different from other tests with which you are familiar. First, the Smarter Balanced assessments are computer adaptive tests and take advantage of technology to customize each test to the individual student. This means that when a student answers a question correctly, the next question the student sees will be slightly more difficult. Likewise, if an answer is incorrect, the next question will be somewhat less difficult. This procedure allows for more efficient and precise measurement of students' knowledge and skills. Second, the assessment is a criterion-referenced test, meaning that it reports individual students understanding of a particular set of academic knowledge and skills. As a result, scores are not meant to compare students to each other, but rather to determine how well each student understands the content assessed. The Interim Information popup contains specific information about the nature of the interim assessment and suggests appropriate ways to interpret interim assessment results. The text is as follows: Important Information about Interim Assessments 1. Questions on interim tests that require human scoring may be scored locally by teachers. This local scoring is not subject to the rigorous controls used in summative assessments and local results may show some variations. 2. Interim assessment questions are not secure. Exposure to, and familiarity with, test questions may affect student performance and the accuracy of interim results. The legend provides a key to the visual elements of the report. It appears as Figure of 61- Updated April 21, 2015

19 Figure 4. Individual Student Report Legend IA Reporting System User Guide Printing Clicking the Print icon allows users to print a report in PDF format in color or grayscale. The printed report contains all visible content from the report on one page and Report Information and Legends are provided on a second page. Score Displays As shown in Figure 5, the top right corner of the report displays mathematics and ELA/literacy overall scale scores and Achievement Levels for the interim assessment selected. -19 of 61- Updated April 21, 2015

20 Figure 5. Individual Student Report Overall Scores IA Reporting System User Guide Under each subject header, the report displays the student s score and Achievement Level on a colored band representing the range of possible scores and the achievement level scores, as shown in Figure 6. If a student has taken the interim assessment in only one content area, the overall score and achievement level for that content area will appear under the student s name. The Achievement Level is accompanied by an Achievement Level Descriptor (ALD). The student s score is represented by a number and a line marking a spot on the horizontal scale, surrounded by brackets indicating error bands. These error bands represent the range of scores a student might have received had the student taken the test multiple times. Figure 6. Individual Student Report Overall Score Bar Below the overall score display, the report displays the student s Claim Level (Below Standard, At/Near Standard, Above Standard) for each of the subject s Claims, as shown in Figure 7 and Figure 8. The display contains descriptions of what each Claim measures and what performance at a specific level indicates about a student s mastery of the standards. -20 of 61- Updated April 21, 2015

21 Figure 7. Individual Student Report Mathematics Claim Levels Figure 8. Individual Student Report ELA/literacy Claim Levels Reported Accommodations Where appropriate, the report (Figure 9) displays Accommodations status messages based on the eligibility of the student (if known), availability of the accommodation (if known), and use of accommodation (if known). Changes to test setting not captured in the Test Order Management System (TOMS) are captured in the without prior registration category below. For interim assessments, test settings can be changed right before giving the assessment by the test administrator without submitting them through TOMS. -21 of 61- Updated April 21, 2015

22 Figure 9. Individual Student Report Accommodations Reporting The accommodations reported are the test settings set by the TA right before testing or through TOMS. The actual use of the available accommodations is not captured at test time and will not be displayed by the IA Reporting System. Language Support The textual content of the IA Reporting System is available for viewing and printing in English by default, but users will soon be able to select to view or print the content in either Spanish or Vietnamese. 4.2 List of Students in a Grade Report This report provides assessment results for a list of students. The list can be viewed either with mathematics and ELA/literacy side by side, or in a subject detail view that shows Overall scale scores complemented by Claim Levels (see Section 7.4 Claims Reporting for more information on Claim Levels). Figure 10 shows the variety of ways to sort this list of students within a grade. Please note: The grade level displayed on this report is the grade level of the interim assessment and will not necessarily correspond with the grade level in which the student is enrolled. -22 of 61- Updated April 21, 2015

23 Figure 10. List of Students in a Grade Report IA Reporting System User Guide of 61- Updated April 21, 2015

24 4.2.1 Report Features Breadcrumbs: At the top of the page, users can navigate back to the school, district, and state level reports. Report Information: Detailed report information regarding the report s purpose and its prospective uses is accessible by pausing on or clicking the i information icon next to the report s title. Download: The Download button allows users to download the current page as a comma separated value (CSV) file or to download the Individual Student Report of the listed students. Assessment / Academic Year Selector: A drop-down list of all assessments that the students have taken, organized by assessment type and date administered. Dropdown also allows selection of other academic years for which there is available assessment data. Filter: The Filter button opens a window that enables advanced filtering of student results by student demographic information. Select a Student: Checkboxes next to student names allow users to select specific students for comparison or printing. Deselect all removes the selection. Column Sorting: Column headers allow users to sort the list of students by the values in each column. Overall Score and Error Bands: The overall ICA score is indicated by a single number and as a dot on a continuum of scores. The score is surrounded by error bands, marked with a gray bar. This range indicates the range of scores that the student would be likely to achieve even if he or she took the test multiple times. Claim Information: Mathematics and ELA/literacy are composed of claims. Within each claim, student performance is reported as Below Standard, At/Near Standard, or Above Standard. A description of each claim is provided Detailed Descriptions Navigation Navigating the Grade Reports is the same as navigating for Individual Student Reports Selecting Assessment Results A user can see results of any assessment with available results via the Assessment Selector (shown in Figure 11). The options in the Assessment Selector are based on all results for assessments designed for that grade. If a student has no results for the selected assessment, his or her row is not displayed. If a student has results only for either mathematics or ELA/literacy, then he or she is displayed when viewing both subjects, but is not displayed for the subject where there is no result. This list does not represent enrollment-based classes or sections, as there is no persistent educator-student links in the Data Warehouse. However, students in the list can be filtered, -24 of 61- Updated April 21, 2015

25 sorted, and selected to assist in narrowing focus. Filters are provided using all demographic and program data available. The list can be sorted in ascending or descending order by the values in any column (e.g., alphabetically by last name or by overall scale scores). Figure 11. Assessment Selector for a List of Students in a Grade When a previous academic year is selected, the report displays a warning message alerting users that the students displayed are those who took the selected grade s assessment in a prior academic year (as shown in Figure 12). Figure 12. Previous Year Warning By default, the report displays an Overview with results from both mathematics and ELA/literacy assessments. Selecting either the Mathematics or ELA/Literacy button displays a view that shows overall results and Claim Levels for the selected assessment. Figure 13 shows the result when the Mathematics button is selected. -25 of 61- Updated April 21, 2015

26 Figure 13. Mathematics Overall and Claim Scores IA Reporting System User Guide Informational Popup Windows Most popup information is the same as for Individual Student Reports. The legend provides descriptions of all pictorial elements of the report, including Achievement Levels, Error Bands, Claim Levels, and the scale bar. It appears as Figure 14 below. Figure 14. Legend for List of Students in a Grade -26 of 61- Updated April 21, 2015

27 Downloads Download options are available by clicking the Download icon. There are several options for downloading information through the List of Students in a Grade report, as shown on Figure 15. Figure 15. Download Options Window Locating Specific Students 1. Current View: All users with access to this report can download a tabular version of the current view of summary data (students, scores, and achievement levels, including any active filters, sorting, or selection) in CSV format for accessibility or external analysis. 2. Student Assessment Results: A user with appropriate permissions can download the underlying Student Assessment Results for the students displayed on the report. 3. Printable Student Reports: A user with appropriate permissions can download a batch PDF of Individual Student Reports for the students displayed on the report. A user can use the Sorting, Filtering, and Find features to locate specific students, and the Selection feature to narrow focus onto any selected students. Please note that users assigned with the PII role will be able to access these functions. Sorting Sort the table by any column ascending or descending: alphabetically by student last name, by Overall scale scores, or by Claim Levels. An example of a grade sorting is shown in Figure 16. Figure 16. Sorting for List of Students in a Grade Filtering Filter the student list by student attributes Selecting multiple options in a filter returns a list composed of any students in either group. Selecting options in more than one filter (e.g., Gender, female and Migrant Status, yes) returns a list of students who fit into both groups. Find a Student A user can click the Find box or use the browser Find command (Ctrl+F), to type in a part or whole name of a student. The report scrolls down to the first entry matching the search string and provides controls to navigate through any matching entries. Select Specific Students Each student in the list has a checkbox next to his or her name to narrow the list. A user can select specific students to compare them to one another or print them out in a batch. An example is shown in Figure of 61- Updated April 21, 2015

28 Figure 17. Narrowing the List of Students in a Grade IA Reporting System User Guide Printing The report as displayed can be printed directly by the built-in browser Print command (Ctrl+P). The List of Students in a Grade report also provides the ability to download the underlying Individual Student Reports in a batch PDF file for bulk printing (see Downloads). Language Support The textual content of the IA Reporting System is available for viewing and printing in English by default, but users will be able to select to view or print the content in either Spanish or Vietnamese. To use this function, on the list of students interface, select the checkboxes next to the students names. Select the Download icon. Choose the Printable Student Reports option. Please refer to Section 5.2 Printable Student Reports for more information. -28 of 61- Updated April 21, 2015

29 5.0 DATA DOWNLOADS IA Reporting System User Guide In addition to the predesigned reports, the IA Reporting System offers authorized users (see Primary Audiences section below) the ability to download data for distribution or further review and analysis in external systems. This section describes the different types of downloads in detail. 5.1 Student Assessment Results Description This is a bulk download of the assessment results, with one row per student assessment. The resulting files contain all of the data for Overall and Claim scores (e.g., scale score, error band, level determination), as well as all the student data (e.g., demographics, grade/school/district/state attribution) Primary Audiences This download is primarily intended for LEA administrators and school administrators. Authorized users must be assigned the SAREXTRACTS user role by their LEA CAASPP coordinator. LEA CAASPP coordinators should carefully consider their LEA s data policies when assigning the SAREXTRACTS user role to other authorized users such as teachers or other educators. The data extract will give the user the ability to import data into local student information systems or to generate summary level information. Use caution when analyzing and disseminating the data as the interim assessments may have been administered to students under different conditions, even within the same school or classroom. Additionally, consider the conditions under which the assessments were administered prior to importing them into any student information systems. Navigation Users who are authorized and have the proper permissions can request this download through the Download interface. Features This download is supported at the District, School, and Grade level; the two lowest levels are executed synchronously, while the larger downloads for district leverage the HTTPS Pickup Zone. This download request generates file pairs for each assessment, where an interim comprehensive assessment is defined by the Academic Year, Grade, Subject, and administration window. The file pair is composed of 1) a large CSV containing all of the assessment results data directly reported on by the IA Reporting System and 2) a JavaScript Object Notation (JSON) metadata file that includes all repetitive data common to all assessment results. Data Summary The Student Assessment Results data specification format for the California Interim Reports is published in detail to: of 61- Updated April 21, 2015

30 5.2 Printable Student Reports IA Reporting System User Guide Printable Student Reports are available only from the List of Students in a Grade report web page. This download option Purpose is available on the upper left of the interface, under the Breadcrumbs. The purpose of this download is to provide educators with the ability to download and print batches of individual student reports at once. This feature enables a school administrator to print out an entire batch of individual student reports for an interim assessment at a particular grade level. For ICA results, a student s report is approximately three pages one page for mathematics, one page for ELA/literacy, and one page for reference material (e.g. Report Information and Legend). For IAB results, a student s report is approximately 2 pages one page for the block assessed and one page for reference materials. Navigation A user would select the Printable Student Reports option from the Download popup window, as shown in Figure 18. The system then displays a second popup window (as seen in Figure 19) that requires users to specify whether they want to print out in grayscale or color and whether they want the reports to print out in English, Spanish, or Vietnamese. The translated student report will be available soon. Figure 18. Download Popup Window Figure 19. Download Printable Student Report Options A status window will appear that either tells users that an will notify them when the file is ready (if the system must generate a large set of student reports), as seen in Figure 20, or provides them with a secure URL at which to download it, as seen in Figure 21. This for the first option will generate after a period of 24 hours. -30 of 61- Updated April 21, 2015

31 Figure 20. Download Confirmation Window Figure 21. Download Printable Student Report Options Clicking the URL allows the user to download a zip file with the requested individual student reports in PDF format. When the user opens the file, the first page includes the name of the school, grade, number of pages, number of students, name of requestor, and date. An example is shown in Figure 22. Figure 22. Cover Page for Printable Student Reports Size Constraints In the event that a single batch request contains more than 1,000 students, it is divided into separate files that are made available to the user to download. Each PDF starts with a cover page that includes all the above information, but also indicates how the system has divided the request into files, the place of each file within the larger batch (e.g., Part 1 of 3), and the names of the first and last student in this file (see Figure 23). -31 of 61- Updated April 21, 2015

32 Figure 23. Segments Cover Page for Printable Student Reports -32 of 61- Updated April 21, 2015

33 6.0 REPORTING FEATURES SUMMARY Table 3. Reporting Features Summary Feature Assessment Accommodations Claim Level Reporting Context Security Download: Current View as CSV Download: Student Assessment Results Filtering Find a Interim Reporting Legend Navigation Description IA Reporting System User Guide Display of Accommodations for which a student was eligible, and whether they were available or used For all users, reporting on mathematics and/or ELA/literacy subscores for ICA For all users, access to features and PII is enforced via the Smarter Balanced Single Sign-on and Permissions components For all users, download the current view of summary data (including any active filters, sorting, or selection) in a nonvisual tabular CSV format For users with permissions to use this feature, a request to download the student assessment results (including test, student, and scoring data) For all users, the ability to disaggregate results by student attributes For all users, the ability to search for a specific character string (e.g., name or partial) within the current report For users with permissions to view PII, reporting on overall mathematics and/or ELA/literacy scores for the ICA and IAB For all users, summaries and description of visual design elements in a given report Login/logout User name display Language selection Breadcrumb/hierarchy navigation User guide Frequently asked questions Glossary Resources links to keyword-driven content searches for relevant materials in the digital library Legend opens popup with legend text Individual Student Report List of Students in a Grade Sorting Reordering list elements by any column -33 of 61- Updated April 21, 2015

34 7.0 DEFINITION OF TERMS 7.1 Accessibility Features of the Reports All reports are accessible in the following manners. Screen readers and keyboard navigation All reports are usable via screen reader and accessible to keyboard navigation, using standard solutions for accessibility (e.g., tab-stop ordering and semantic alt-text for images). Text-to-speech, braille, screen-reading software Reports will follow guidelines for content to be used by screen-reader and support refreshable braille, text-to-speech tags, and text magnifying software. Braille support must include contracted, un-contracted, and Nemeth Braille. In general, text-to-speech and braille are handled by client-side solutions, although Amplify has included design and code elements (e.g., tab-stops and semantic alt-text) that support a user experience that is 508-compliant. The target operating systems and browsers have many options for supporting Text-to-Speech and Braille. The American Foundation for the Blind provides guidance on supporting screen-reading software and braille-related extensions here: Magnification/Zoom All supported browsers provide native zoom scaling from 25% to 500%. There are also several freely-available extensions or plugins for each browser to increase text size or enlarge images. Highlighter Supported browsers provide native select-to-highlight which can be configured with an array of color choices and behaviors to support contrast needs. Each browser also supports several freely available extensions or plugins that provide more advanced clientside highlighting capabilities. Color/Contras Supported browsers provide native contrast controls, but freely available extensions or plugins can provide additional support for displaying content in a number of alternative contrast schemes. All color choices (including hue, saturation, and contrast) have been verified as meeting visual acuity standards. 7.2 Accommodations In the Smarter Balanced Usability, Accessibility, and Accommodations Guidelines, accessibility features are identified across three categories, Universal Tools, Designated Supports, and Accommodations. If a student is eligible for access to or use of one or more Accommodation(s), then this information is captured in the Data Warehouse and displayed on the Individual Student Report -34 of 61- Updated April 21, 2015

35 only. Information on Universal Tools and Designated Supports are not included in the Individual Student Report. Accommodation information is displayed as follows: Figure 24. Accommodations Display Additional information on Universal Tools and Designated Supports is included in the Audit Extensible Markup Language (XML) for future use, but is not reported on in the current version of the IA Reporting System. 7.3 Achievement Levels and Proficiency Achievement Levels are defined and described by the Consortium. Defining these levels of achievement is a reporting feature that is federally required under the No Child Left Behind Act. Smarter Balanced has also developed a set of initial, policy achievement level descriptors (ALDs) for English language arts/literacy (ELA/literacy) and mathematics that are aligned with the Common Core State Standards (CCSS) and the Smarter Balanced assessment claims. The purpose of these descriptors is to specify, in content terms, the knowledge and skills that students display at the four levels of achievement (i.e., Level 1, Level 2, Level 3, and Level 4). Table 4 shows the interim assessment achievement levels and their corresponding achievement levels on the summative assessments. Table 4. Crosswalk of the Interim Assessment Achievement Levels to the Summative Assessment Achievement Levels Interim Assessment Achievement Level Level 1 Level 2 Level 3 Level 4 Summative Assessment Achievement Level Standard Not Met Standard Nearly Met Standard Met Standard Exceeded Although the Achievement Level Descriptors are intended to aid interpretation of Achievement Levels, they will be less precise than scale scores for describing student gains over time or changes -35 of 61- Updated April 21, 2015

36 in achievement gaps among groups, since they do not reveal changes of student scores within the bands defined by the achievement levels. Furthermore, there is not a critical shift in student knowledge or understanding that occurs at a single cut score point. Thus, the achievement levels should be understood as representing approximations of levels at which students demonstrate mastery of a set of concepts and skills, and the scale scores just above and below an achievement level as within a general band of performance. A student is considered proficient if test results place his or her score in Level 3 or 4, and therefore the Level 3 cut-score is somewhat analogous to a proficiency line. 7.4 Claims Reporting Smarter Balanced has identified the following Claims that support the Overall subject (also known as Composite ) scores, as seen in Table 3. Table 5. Claims List Subject Mathematics ELA/Literacy Claims List Concepts and Procedures Problem Solving and Modeling & Data Analysis Communicating Reasoning Reading Writing Listening Research and Inquiry Reporting provides Claim-level insights for individual students, and for the List of Students in a Grade report, but Claim-level aggregations are not supported. Claim Calculation Details and Levels Determination There is a single score per claim, which derives from the same scale as the Overall Composite score. The width of Category 2 for each claim is based on the same amount of Standard Error of Measurement (SEM). As such, the actual widths for the Category 2 scores across different claims vary as the SEMs of the claims vary. Any claim score lower than the bottom end of the error band around the Composite L2/L3 cut score (a.k.a. the proficiency line ) is considered a Category 1 claim score. Any claim score within the error bands is considered a Category 2 claim score. Any claim score above the upper end of the error band around the Composite L2/L3 cut score is considered a Category 3 claim score. -36 of 61- Updated April 21, 2015

37 Figure 25. Claims Score Illustration In Category 3, there is confidence that the student is on track (i.e., the student should continue receiving current levels of support at this point in time). In Category 1, there is confidence that the student is not on track (i.e., the student is clearly in need of help, support, or assistance; the student is clearly not demonstrating the expected knowledge or ability to apply skill). In Category 2, the student is not clearly in either category and their status cannot be determined with an appropriate level of confidence. Claim Levels and Icons The following labels for the List of Students in a Grade and Individual Student Reports have been presented to all governing state leads for feedback have been approved by Smarter Balanced: The assessment results indicate that the student understands and can apply their knowledge to the standards in this subject area for their grade. The assessment results may be just above or just below the standard, but due to the error band (see above), it is impossible to determine with a sufficient degree of confidence. The assessment results indicate that the student has not yet met the standard in this subject-area for their grade. Sorting Behavior In the List of Students in a Grade report, sorting by claim scores orders by the underlying claim scale scores, even though those scores are not exposed in the visual report. Claim scale scores are not exposed via any Current View download, but are included in the Student Assessment Result download. Assumptions 1. For the Individual Student Report and List of Students in a Grade reports, the IA Reporting System displays Claim Levels as visual icons with labels and descriptors, but does not expose the actual numerical claim score data. -37 of 61- Updated April 21, 2015

38 2. Every assessment result provided to the Data Warehouse includes a student s scale score, level determination, scale maximum, scale minimum, scale units, and error band. Claim scale scores are measured on the same scale as the Overall score. Claim error bands vary by Claim. 3. Smarter Balanced provides the algorithm for calculating the claim score groupings per grade, subject and claim to the Score Integration vendor; neither the Data Warehouse nor the IA Reporting System dynamically calculates scale scores, error bands, or level determinations. -38 of 61- Updated April 21, 2015

39 7.5 Context Security, Access Patterns, and Single Sign-On (SSO) Registered users can see: any PII-protected data that they are authorized to view, as managed in permissions at the level of school (e.g., an educator in school X with PII access can see students in school X) All users must have login credentials for the SSO component and must log in to view reports. There is no anonymous public user support (unregistered users, logged-out users). All Single Sign-On functionality is managed by the Smarter Balanced SSO shared service. The Smarter Balanced Data Warehouse and Reporting System receive user authorizations from the Smarter Balanced SSO shared service. Neither the Smarter Balanced Data Warehouse nor the IA Reporting System maintain or verify user authorizations. The Smarter Balanced SSO shared service provides the interface for management of associations between users and student PII for the IA Reporting System. Access to Reports and PII are managed by the state to provide access to LEA CAASPP coordinators. The LEA CAASPP coordinator is to provide access to the IA Reporting System to authorized users within their LEA via the Smarter Balanced Shared Services. Provisioning and management of Consortium user accounts are supported by the Consortium via the Smarter Balanced Shared Services. Educators may have access to student PII for their associated entities (school(s)/district(s)), which can be controlled within the SSO shared service at the school level. The educator s LEA CAASPP coordinator is to provide this access. Non-educators have no access to student information; the Consortium does not support parent or student login in the Consortium Data Warehouse and Reporting systems. Permissions are available on a per-feature basis, and the Permissions component allows for each permission to be applied to one or more nodes in the hierarchy (e.g., Student PII Access for School X, Student PII Access for District Y). Only the permissions used for California are listed here. Student Assessment Results Extract Student personally identifiable information (also known as PII Access ) 7.6 Error Bands and Standard Error of Measurement Standard Error of Measurement (SEM) is accounted for in the reports by the error bands that are displayed in the Individual Student Report and the List of Students in a Grade report for overall scores. SEMs are also included on claim scores in the Student Assessment Results downloads. The error band meaning, as seen in Figure 26, is explained in the Legend of each report that displays them: Smarter Balanced tests provide the most precise scores possible within a reasonable time limit, but no test can be 100 percent accurate. The error band indicates the range of scores that students would very likely achieve if they were to take the test multiple times. It is similar to the margin of error that newspapers report for public opinion surveys. -39 of 61- Updated April 21, 2015

40 Figure 26. Error Band Illustration 7. 7 Interim Assessment Blocks (IAB) Reporting The IABs focus on a smaller set of skills and are designed to provide targeted information for educators about student performance. Educators may use the IAB results to focus classroom instructions on certain areas where the student(s) need improvement. Results will be reported at the Claim Level. Claims for ELA/literacy pertain to reading, writing, listening/speaking, and research. Find further description of the ELA reporting achievement level descriptors in the table below. The second time a student takes the same interim assessment, the results are sent in with the same name and date but do not replace the original assessment report; previous results are not deleted. (Previous results will persist to provide a comparison between the first test opportunity and the second.) Table 6. Claim Level Descriptors The assessment results indicate that the student understands and can apply their knowledge to the standards in this subject area for their grade. The assessment results may be just above or just below the standard, but due to the error band (see above), it is impossible to determine with a sufficient degree of confidence. The assessment results indicate that the student has not yet met the standard in this subject-area for their grade. 7.8 Interim Comprehensive Assessments (ICA) Reporting From a reporting perspective, ICA results are reported through the Individual Student Reports (with overall and claim scale scores for each subject) and through the List of Students in a Grade report. From a policy perspective, an ICA is intended for use as a benchmarking tool, where administration requirements are less stringent than for the summative assessment. Consequently, summative results support higher-level aggregation (between grades, schools or districts) because the test conditions and administration requirements are comparable. ICAs, which may be administered with greater variations in test conditions, would introduce validity issues if aggregated and are therefore only reported on a per student basis. See the descriptors of an ICA score in Table 6 (ELA) and Table 7 (Mathematics) below. -40 of 61- Updated April 21, 2015

41 Table 7. ELA/literacy Reporting Achievement Level Descriptors English language arts/literacy Reporting Achievement Level Descriptors High School Grades 6 8 Grades 3 5 Level 4 Level 4 Level 4 The student has exceeded the achievement standard and demonstrates the knowledge and skills in English language arts/ literacy needed for likely success in entry-level credit-bearing college coursework after high school. The student has exceeded the achievement standard and demonstrates advanced progress toward mastery of the knowledge and skills in English language arts/literacy needed for likely success in entry-level creditbearing college coursework after high school. The student has exceeded the achievement standard and demonstrates advanced progress toward mastery of the knowledge and skills in English language arts/literacy needed for likely success in future coursework. Level 3 Level 3 Level 3 The student has met the achievement standard and demonstrates progress toward mastery of the knowledge and skills in English language arts/ literacy needed for likely success in entry-level credit-bearing college coursework after completing high school coursework. The student has met the achievement standard and demonstrates progress toward mastery of the knowledge and skills in English language arts/ literacy needed for likely success in entry-level credit-bearing college coursework after high school. The student has met the achievement standard and demonstrates progress toward mastery of the knowledge and skills in English language arts/ literacy needed for likely success in future coursework. Level 2 Level 2 Level 2 The student has nearly met the achievement standard and may require further development to demonstrate the knowledge and skills in English language arts/ literacy needed for likely success in entry-level credit-bearing college coursework after high school. The student has nearly met the achievement standard and may require further development to demonstrate the knowledge and skills in English language arts/ literacy needed for likely success in entry-level credit-bearing college coursework after high school. The student has nearly met the achievement standard and may require further development to demonstrate the knowledge and skills in English language arts/ literacy needed for likely success in future coursework. Level 1 Level 1 Level 1 The student has not met the achievement standard and needs substantial improvement to demonstrate the knowledge and skills in English language arts/ literacy needed for likely success in entry-level credit-bearing college coursework after high school. The student has not met the achievement standard and needs substantial improvement to demonstrate the knowledge and skills in English language arts/ literacy needed for likely success in entry-level credit-bearing college coursework after high school. The student has not met the achievement standard and needs substantial improvement to demonstrate the knowledge and skills in English language arts/ literacy needed for likely success in future coursework. -41 of 61- Updated April 21, 2015

42 Table 8. Mathematics Reporting Achievement Level Descriptors Mathematics Reporting Achievement Level Descriptors High School Grades 6 8 Grades 3 5 Level 4 Level 4 Level 4 The student has exceeded the achievement standard and demonstrates the knowledge and skills in mathematics needed for likely success in entry-level creditbearing college coursework after high school. The student has exceeded the achievement standard and demonstrates the knowledge and skills in mathematics needed for likely success in entry-level creditbearing college coursework after high school. The student has exceeded the achievement standard and demonstrates advanced progress toward mastery of the knowledge and skills in mathematics needed for likely success in future coursework Level 3 Level 3 Level 3 The student has met the achievement standard and demonstrates progress toward mastery of the knowledge and skills in mathematics needed for likely success in entry-level creditbearing college coursework after completing high school coursework. The student has met the achievement standard and demonstrates progress toward mastery of the knowledge and skills in mathematics needed for likely success in entry-level creditbearing college coursework after high school. The student has met the achievement standard and demonstrates progress toward mastery of the knowledge and skills in mathematics needed for likely success in future coursework. Level 2 Level 2 Level 2 The student has nearly met the achievement standard and may require further development to demonstrate the knowledge and skills in mathematics needed for likely success in entry-level creditbearing college coursework after high school. The student has nearly met the achievement standard and may require further development to demonstrate the knowledge and skills in mathematics needed for likely success in entry-level creditbearing college coursework after high school. The student has nearly met the achievement standard and may require further development to demonstrate the knowledge and skills in mathematics needed for likely success in future coursework. Level 1 Level 1 Level 1 The student has not met the achievement standard and needs substantial improvement to demonstrate the knowledge and skills in mathematics needed for likely success in entry-level creditbearing college coursework after high school. The student has not met the achievement standard and needs substantial improvement to demonstrate the knowledge and skills in mathematics needed for likely success in entry-level creditbearing college coursework after high school. The student has not met the achievement standard and needs substantial improvement to demonstrate the knowledge and skills in mathematics needed for likely success in future coursework. 7.9 Assumptions 1. On the List of Students in a Grade report and Individual Student Report, educators see a dropdown list of all assessments taken by that student or group of students. 2. Assessments are listed as YYYY.MM.DD - Grade X - Type - Subject (e.g., Grade 4 - Interim - Mathematics). 3. The second time a student takes the same interim assessment, the results are sent in but do not replace the original assessment report. Previous results will persist to provide a comparison between the first test opportunity and the second. -42 of 61- Updated April 21, 2015

43 4. The list is not filtered and displays all assessments that have been loaded into the Data Warehouse. 5. On the List of Students in a Grade report, any assessment given to at least one of the students appears on the list. Students with no results for that assessment do not appear when that assessment is selected. 6. The following qualifications were endorsed by Consortium Leadership on July 16, 2014, and are accessed on the List of Students in a Grade and Individual Student Report by pausing on or clicking the blue information icon : There is no reporting for interim comprehensive assessments above the level of the List of Students in a Grade for a specific grade in an individual school (no grade, school, or district comparison reports). 7. All interim comprehensive assessment language/content for Qualifications or Descriptions have been provided by either the Test Design and Validation Work Group, Lead Psychometrician, or the 05-Interim Assessment Work Group, subject to any specifications from the Reporting vendor Help General help information and links to several sets of resources: Important Information about Interim Assessments 1. Questions on interim tests that require human scoring may be scored locally by teachers. This local scoring is not subject to the rigorous controls used in summative assessment and local results may show some variations. 2. Interim assessment questions are not secure. Exposure to and familiarity with test questions may affect student performance and the accuracy of interim results. Frequently asked questions (currently in development) Glossary (currently in development) Resources (links to keyword-driven search of the Digital Library) 7.11 HTTPS Pickup Zone For download requests that are either 1) too large for direct download via the browser or 2) complex enough to require significant processing time, the download is made available via an HTTPS Pickup Zone. Once the files have been generated and are available for pickup, the user can retrieve the file(s) from the URL provided to them. The download is only retrievable from the provided URL by the original requesting user. The Pickup Zone relies on the authentication of the Smarter Balanced Single Sign-On component to validate that the appropriate user is accessing the file, and then securely delivers the file to the user via encrypted transmission using HTTPS. There are no additional configurations or tools needed for a user to access the Pickup Zone other than the browser they are using to view reports and request downloads. -43 of 61- Updated April 21, 2015

44 7.12 Language Support IA Reporting System User Guide The textual content of the IA Reporting System is available for viewing and printing in English by default, but users can select alternative languages, including the Spanish and Vietnamese translations Legends A Legend popup is available on each reporting page. It provides explanations for icons, color choices and other design elements of the reports Logging The IA Reporting System logs any access to confidential student data. Logs can be accessed by technical staff with appropriate permissions. Instructions are included in technical documentation (Runbooks). The IA Reporting System logs the user s information and the activities that the user performed with the confidential data Navigation Landing Page Any user who is not logged in sees a landing page with a link to log in and a small number of links to public resources. See Figure 27 to view the landing page. Figure 27. Reporting Landing Page (for logged out users) -44 of 61- Updated April 21, 2015

45 IA Reporting System User Guide Breadcrumbs All reports provide a breadcrumb trail to maintain user understanding of their current location, where each step in the trail is clickable to jump to that page to navigate through the report hierarchy. Figure 28. Breadcrumb Navigation System provides links to public Digital Library Resources via the Resources button. Navigation is assessment-data driven, and the system cannot provide navigation to reports for any entities that do not have assessment data. Conversely, any student record s hierarchy generates the reporting hierarchy; i.e., if a single student s assessment outcome exists, then that student s grade, school, and district are navigable to enable access to that student s Individual Student Report. System has coherent navigation to and from reports and features, as well as a consistent user experience across stakeholders and reports. System does not provide jump-navigation to specific reports for any users based on account information or preferences Permissions Permissions are available for states, districts, and schools to manage access rights to features and information. Permissions are managed through the Smarter Balanced Permissions component. For Reporting, the permissions available are feature-based, rather than role-based, and must be limited in scope to one or more schools, districts, or states. The features covered by permissions are: Assessment Completion Extract Audit XML Extract Individual Item Response Data Extract Student Assessment Results Extracts (also known as SAREXTRACTS ) Student personally identifiable information (also known as PII Access ) Student Registration Statistics Extract 7.17 Printing, Individual and Batch Individual Printing The Individual Student Report has a special printer-friendly output in PDF format to ensure all report and supporting information is included in the printout, and should be suitable for distribution to students, parents, and teachers. All other reports print directly from the browser using the standard Ctrl+P print command. Batch Printing Batch printing is available through the IA Reporting System to help distribute printed versions of the Individual Student Report. -45 of 61- Updated April 21, 2015

46 List of Students by Grade-level Test Administered IA Reporting System User Guide The authorized user can request a batch print of the Individual Student Report for all of the students included in the current view. If a user has filtered or selected some subset of students, then the batch print includes only those visible. Batch Printing from a School-Level View of Students The authorized user can request a batch print of the Individual Student Report for all of the students in the school or a subset of the school (e.g., certain grades, filtered list). If a user has filtered to a specific subset of students, then the batch print includes only those students who meet the limiting criteria. To make it easier for users to download and print the Individual Student Reports, the batch printing download is split automatically by grade. So a user who requests a download of all the students from a middle school receives three PDF files, one for each grade. To ensure that reporting system performance is not negatively affected, batch printing is not available at the district level. Download popup window When a user clicks the Download button on the List of Students in a Grade or Results by Grade for a School report, the Download window displays (see Figure 29). From that popup window, users can: 1. Download the current view as a CSV file this downloads a CSV version of the report (including any filters or sorting) into the browser. This allows users to read the data in tabular form. 2. Download the student assessment results this downloads the raw assessment result data (as a CSV file) captured for the interim comprehensive assessment for all the students whose scores are represented in the current report. This choice is only available to users who have PII access to the underlying students. Figure 29. Download Popup Window 3. Download printable Individual Student Reports this downloads the Individual Student Reports for all underlying students in one combined PDF. Selecting this option opens a second popup window (see Figure 30). -46 of 61- Updated April 21, 2015

47 Figure 30. Printable Student Report Options IA Reporting System User Guide Report options popup window When a user downloads the printable student reports from the Download popup window, a second popup window appears. This window allows the user to customize their batch print job as follows: Grayscale (for black and white printing) or color. Grayscale is default. Language Support - to print reports in Spanish or Vietnamese Once the options are selected and the Download button is clicked, the Report URL confirmation window (as seen in Figure 31) opens. Figure 31. Download Confirmation Window Report URL popup window When the user clicks the Download button on the Report Options popup window, a popup window appears. It notifies the user of the URL or URLs (for multiple files) where their file(s) will appear. The user is then able to click the URL to receive their requested file. Once a user initiates a batch-print request, the job is scheduled in a queue, and it may take some time to complete, depending on the size of the queue and other demands on the system. The job includes steps to generate the pages for the selected reports, integrate the individual student reports into one or more large PDF files, and deposit the PDF files to the HTTPS Pickup Zone. For a batch-print request made from a List of Students in a Grade report, all selected students are included, and their reports are arranged alphabetically by last name in the large PDF files. At this -47 of 61- Updated April 21, 2015

48 level, most batch-print requests are less than 1,000 students in a single grade, and therefore likely contained in a single large PDF file. If there are more than 1,000 students, the batch-print is split into multiple PDF files. For a batch-print request made from the Results by Grade in a School report, PDFs are generated separately for each grade, which results in at least one large PDF per grade. If a given grade has more than 1,000 students, then that grade is split into multiple PDF files. The PII Access permission extends to the Batch Printing feature. If users do not have PII Access permissions, they do not have the batch print option available. If users are authorized to see the List of Students in a Grade report, they have PII Access. However, a user without PII access rights may have access to the Results by Grade for a School report; in that case, the user does not see the batch-print download option Security The IA Reporting System automatically logs users out after a configurable specified number of minutes with no activity. The period is set by the SSO component and is currently assumed to be 30 minutes Scale Score All reporting is based on scale scores produced by the Scoring component of the Platform. Each subject (mathematics and ELA/literacy) has an overall Composite score and a small number of Claim scores calculated in the Scoring component and integrated by Test Integration into the test XML. Overall scale scores are displayed on the Individual Student Report and the List of Students in a Grade Report, while Claim scale scores are only displayed through data extract. The scoring integrator component also calculates any Achievement Levels, Categories, Claim Levels, and Error Bands Section 508 Compliance The IA Reporting System is in compliance with accessibility requirements specified in Section 508 Amendment to the Rehabilitation Act of 1973, specifically the following subsections: Software Applications and Operating Systems Web-based Intranet and Internet Information and Applications Performance Criteria Information, Documentation, and Support Amplify has reviewed the system to ensure that information is equitably accessible. Individual Student Report The Individual Student Report has a special PDF print capability, but does not provide a nonvisual/tabular download. The report can be navigated using the keyboard and accessed in screen readers. The visual design elements are rendered in sufficient color contrast List of Students in a Grade Report For a List of Students in a Grade Report, the user must have access to PII to view the report; therefore anyone who can see this report can download either or both of the following extracts: -48 of 61- Updated April 21, 2015

49 1. Download Current View as CSV This provides the user with a full extract of the assessment outcomes only for the students that are visible at the time, as it takes into account the state of all sort, filter, and selection options. a. For Mathematics and ELA/literacy view, this CSV file provides the Overall scale score, Overall error band, and Overall achievement level data for both subjects for each student. b. For Subject-Detail view, this CSV file provides the Overall scale score, Overall error band, Overall achievement level, and the Level (1, 2, 3) and Label (Below, At/Near, Above Standard) for each Claim in the selected subject. Figure 32. List of Students in a Grade, Download Current View as CSV example 2. Download Student Assessment Results This provides the user with a full extract of the assessment outcomes for all the students that took the interim assessment for that particular Grade. -49 of 61- Updated April 21, 2015

50 8.0 TECHNICAL REQUIREMENTS IA Reporting System User Guide Reports are designed to be viewed using standard up-to-date browsers running on most available hardware: Chrome version 31 or higher (Windows and Mac) Firefox version 26 or higher (Windows and Mac) Internet Explorer 9 or higher (Windows only) Safari 7 or higher (Mac only) Otherwise for IA Reporting System users, the technical requirements for viewing reports do not exceed or extend the technology readiness requirements established by Smarter Balanced. -50 of 61- Updated April 21, 2015

51 9.0 DOCUMENTED ASSUMPTIONS AND CLARIFICATIONS Truth at Test Time The IA Reporting System provides snapshots of student performance at a moment in time. Reports are organized by the information available provided by the Test Delivery System at the time of assessment, including student information and attribution information. Testing hypothetical situations or extrapolations are not supported. Data-Driven Hierarchy Individual student records drive all the reports, and the reporting hierarchy is derived from those records. Any attribution or grouping information is provided at test time by the Test Delivery System, and is not managed or maintained in the Data Warehouse or via the IA Reporting System. Enrollment The Data Warehouse and Reporting System solution is an assessment reporting system and does not have an operational or transactional data store to support reporting on enrollment. It may be used in conjunction with a system that does. Instructional Use The Digital Library is a good source for additional tools and modules that help address instructional questions that may be raised by inferences supported by assessment results. Statistical Analysis The Data Warehouse contains sufficient information to support statistical analysis, and the download/extract capabilities provide the ability to export data to more fully-featured statistical solutions such as SPSS, SAS, Minitab, R, or Microsoft Excel. The Data Warehouse s star schema structure supports integration with OLAP tools. Longitudinal Data and Student IDs All data in the Data Warehouse is implicitly longitudinal, in that all results for a given student are connected by that student s ID. The current version of the system provides reporting designed for one year of data due to additional requirements gathering needed for appropriate display of data over time. Amplify provides visual design services to document direction for future reporting capabilities, including the following year-over-year reports: o Reporting on multiple years of interim comprehensive assessment data for the List of Students in a Grade report o Reporting on multiple years of interim comprehensive assessment data for the Individual Student Report The identity management systems of each state are responsible for reconciling multiple student IDs generated for a single student into a single student ID. The IA Reporting System is dependent on the state identity management systems to provide student IDs. -51 of 61- Updated April 21, 2015

52 Group and Permission Definitions IA Reporting System User Guide Configuration of user groups and permissions are maintained through the Smarter Balanced SSO shared service. Users are associated with school(s)/district(s)/and state(s) by their state. All user associations are passed to the Data Warehouse by the state identification management system. System Functionality Restrictions System functionality in the Data Warehouse and IA Reporting System is granted or restricted to user groups or entities, including Smarter Balanced, state, district, and school building levels by system administrators. Permissions are feature-based. Statistical Support The Data Warehouse and IA Reporting System contain and make available data to allow for: o Conditional Analyses Basic conditional analyses are achievable through filtering of reports. More advanced analyses require external tools applied to data extracted from the Data Warehouse. o Basic Inferential Tests Offline regressions (e.g., psychometric calibration, state/district custom metrics) require the use of external tools applied to data extracted or downloaded from the Data Warehouse. o Frequencies They are available in both the comparing populations and Results by Grade for a School report. o Crosstabs They require the use of external tools applied to data extracted from the Data Warehouse. The Data Warehouse and IA Reporting System do not support: o Descriptive Statistics As per direction of Psychometric validation partner ETS, Amplify recommends against publishing mean or quartile scores for a given aggregation. -52 of 61- Updated April 21, 2015

53 APPENDIX A. REGISTERING USERS AND ASSIGNING PERMISSIONS FOR THE INTERIM ASSESSMENT REPORTING SYSTEM This appendix contains instructions for LEA CAASPP coordinators to provision (i.e., register) authorized users for access to the IA Reporting System. Before assigning any permissions to district or school staff, the LEA CAASPP coordinator should read and carefully consider the information provided in Section 2.0 (Security and Privacy) of the User Guide. Registration of users for the IA Reporting System is completed through the Smarter Balanced Administration and Registration Tool (ART) interface. The instructions in this appendix should be used in coordination with the instructions in the ART User Guide for creating user accounts and the Smarter Balanced Reporting and Data Warehouse Roles. About Managing Users and Their Roles LEA CAASPP coordinators whose LEAs have administered interim assessments will automatically be provided access to the IA Reporting System by ETS. Once given access to the IA Reporting System, the LEA CAASPP coordinator is able to provide access for other authorized LEA personnel. Managing Users. LEA CAASPP coordinators can create user records that are within their LEA. User records cannot be created for a higher level or for an LEA or institution to which they do not belong. For example, if the user belongs to District A, user records can only be created in District A that are associated with the district level or below. Note: The actions described in this section are available to district-, and school-level users. User Roles in the Reporting System The CDE will assign LEA CAASPP coordinators with three user roles: GENERAL, PII, and SAREXTRACTS. Table A.1 describes each of these three roles. LEA CAASPP coordinators will be able to authorize LEA personnel and assign them the appropriate reporting user roles. LEA CAASPP coordinators should consider local policies about disseminating test results. To access the IA Reporting System, authorized users must have the GENERAL role and at least one other reporting roles: IMPORTANT: Access to student PII is strictly regulated by state and federal laws and district policies. -53 of 61- Updated April 21, 2015

54 Table A.1. Roles in the IA Reporting System Role Display Name in ART GENERAL (with Responsible Entity Type as DISTRICT ) GENERAL (with Responsible Entity Type as INSTITUTION ) PII SAREXTRACTS Role Description This role is required for access to the IA Reporting System. This role is required for access to the IA Reporting System. This role provides access to individual student assessment results for all institutions under the associated entities. To activate this role, GENERAL must be assigned as well. This role provides access to download the reported data in detail for all students covered by the institution. To activate this role, GENERAL and PII must be assigned as well. IA Reporting System User Guide Role Example Any user who should have access to view student reports for an LEA (referred to as district in ART), schools in a district, and grades in a school. All authorized LEA personnel must be assigned this user role. Any user who should have access to view student reports for the assigned school, and grades within the school. All authorized school site personnel must be assigned this user role. Any user who should be able to see individual student assessment results. Interim results are only available at a student level, therefore, the LEA CAASPP coordinator must assign the PII role to any additional users. LEA CAASPP coordinators and other LEA staff and educators designated by the LEA CAASPP coordinator with authorization to view student test results through the IA Reporting System should be assigned this role. Note that users with this role may also print an individual student s test results. Any user who should be able to download the assessment results data that drives individual reporting. Only LEA personnel authorized to download the student test results should be assigned this user role as the data file includes all student test results for the institution. IMPORTANT: ART may list other roles not listed here. Only the reporting roles in Table A.1 are used for California. Please note that the only combinations of roles possible are GENERAL and PII or GENERAL, PII, and SAREXTRACTS. The GENERAL and PII combination will grant the user access to individual student assessment results for all institutions under the associated entities. The GENERAL, PII, and SAREXTRACTS combination will allow the user to access AND download the reported data in detail for all students covered by the institution. How the scope of access (i.e. LEA level vs school site level) is determined is covered in the next section. -54 of 61- Updated April 21, 2015

55 Adding the Reporting Roles for an Existing User IA Reporting System User Guide If user has log in credentials to the Smarter Balanced Digital Library, the user s record can be modified in ART to include the reporting roles. If the user does not have Digital Library log in credentials, the user must be added to ART before the reporting roles can be assigned. Refer to the instructions for Adding New Users later in this appendix. Step Action 1 Go to the Smarter Balanced Administration and Registration Tool (ART) portal ( on a Web browser. 2 Log into the ART using your Smarter Balanced log in credentials. These would be the same credentials as your Digital Library log in credentials. 3 Select the [Create/Modify User] icon. 4 On the User Search screen, select the [Search] button to search for a user record. Selecting the [Search] button without including search filters will display all users within your LEA or school. Using the Search Filters will narrow your search results. -55 of 61- Updated April 21, 2015

56 Step Action 5 On the search results table, click on the [ ] button for a user record. The Edit User Information screen will appear. 6 On the Edit User Information screen, select the [ ] button to add a new user role. A new row will appear in the Role Associations area. 7 All users who are authorized to access the IA Reporting System must be assigned the GENERAL reporting role. In the Role column on the new row, select GENERAL from the pull down menu. In the Responsible Entity Type column, select DISTRICT or INSTITUTION. Selecting DISTRICT will provide access to test results for the entire LEA. Selecting INSTITUTION will provide access to test results for the school selected. In the Associated Entity Id column, select the LEA (if Responsible Entity Type is DISTRICT) or school (if Responsible Entity Type is INSTITUTION). 8 Generally, all users should be assigned the PII reporting role to view the interim assessment results. On the Edit User Information screen, select the [ ] button to add a new user role. A new row will appear in the Role Associations area. In the Role column on the new row, select PII from the pull down menu. -56 of 61- Updated April 21, 2015

57 Step Action In the Responsible Entity Type column, select DISTRICT or INSTITUTION. Selecting DISTRICT will provide access to test results for the entire LEA. Selecting INSTITUTION will provide access to test results for the school selected. In the Associated Entity Id column, select the LEA (if Responsible Entity Type is DISTRICT) or school (if Responsible Entity Type is INSTITUTION). 9 Users who have LEA authorization to download a data extract file should also have the SAREXTRACTS reporting role. The data extract file may be used to LEA or school to upload to its local student information system or to generate reports for local use (e.g., roster of test results by classroom). On the Edit User Information screen, select the [ ] button to add a new user role. A new row will appear in the Role Associations area. In the Role column on the new row, select PII from the pull down menu. In the Responsible Entity Type column, select DISTRICT or INSTITUTION. Selecting DISTRICT will provide access to test results for the entire LEA. Selecting INSTITUTION will provide access to test results for the school selected. In the Associated Entity Id column, select the LEA (if Responsible Entity Type is DISTRICT) or school (if Responsible Entity Type is INSTITUTION). 10 Select the [Save] button to save your changes. Adding New Users A user must be added to ART before the reporting roles can be assigned. The user s reporting roles can be added during this process. Step Action 1 Go to the Smarter Balanced Administration and Registration Tool (ART) portal ( on a Web browser. -57 of 61- Updated April 21, 2015

58 Step Action 2 Log into the ART using your Smarter Balanced log in credentials. These would be the same credentials as your Digital Library log in credentials. 3 Select the [Create/Modify User] icon. 4 On the User Search screen, select the [New] button. -58 of 61- Updated April 21, 2015

59 Step Action 5 In the Add User Information section, enter the user s first name, last name, and address in the appropriate fields. Entering the telephone number is optional. For an explanation of these fields, see Table A.2 below. 6 In the Role Associations section, select the [ ] button to add a new user role. A new row will appear in the Role Associations area. 7 In the Role column on the new row, select District Coordinator or School Coordinator from the pull down menu. Users who currently have District Coordinator role can assign new users with the District Coordinator or School Coordinator roles. Users who currently have School Coordinator roles can assign new users with School Coordinator roles. In the Responsible Entity Type column, select DISTRICT or INSTITUTION. Selecting DISTRICT will provide access to test results for the entire LEA. Selecting INSTITUTION will provide access to test results for the school selected. In the Associated Entity Id column, select the LEA (if Responsible Entity Type is DISTRICT) or school (if Responsible Entity Type is INSTITUTION). 8 Select the [ ] button to add the reporting roles: (Each role assignment will require a separate row.) GENERAL (for all users) PII (for all users authorized to view test results) SAREXTRACTS (for users authorized to download a data file of the test results) See Steps 7 9 in the instructions the section above titled Adding the Reporting Roles for an Existing User. -59 of 61- Updated April 21, 2015

60 Step Action 9 RECOMMENDED: Select the [ ] button to add the DL_EndUser role so that the user may also access the Smarter Balanced Digital Library. 10 Select the [Save] button to save your changes. Table A.2. Fields Descriptions for Adding New Users to ART Role Display Name in ART First Name* Last Surname* address* Telephone Number Role* Associated Entity ID* *Required field Role Example User s first name. Up to 35 characters. User s last name. Up to 35 characters. User s address. Up to 128 characters in standard address format. Telephone number including the area code, and optional extension. Up to 24 characters in XXX-XXX-XXXX xnnnnnn. Role assigned to the user. This list is pre-populated. Up to 30 characters. Identifier of entity associated with the user. This list is prepopulated. Up to 20 characters. To add or modify users and user roles using a file upload, please contact the California Technical Assistance Center (CalTAC) Digital Library help desk by phone at or by at caltac@ets.org. Registering Institutions (Schools) All users of the IA Reporting System and the Digital Library must be associated with the district or institution. ETS manages and maintains the district and institution information in ART. LEA CAASPP coordinators should direct questions about registration and requests for changes to school information to the CalTAC Digital Library help desk by phone at or by at caltac@ets.org. -60 of 61- Updated April 21, 2015