Standards-Based Competency Assessments, Grades 3-11

Transcription

1 Standards-Based Competency Assessments, Grades 3-11 Arizona Department of Education TECHNICAL ORIGINAL RFP#: ADED DUE: December 5, 2013 ADE c/o Chief Procurement Officer, Lupita Gomez

2 Lupita Gomez Chief Procurement Officer Arizona Department of Education 1535 W. Jefferson St. Phoenix, AZ ASSESSMENT & INSTRUCTION State Services NCS Pearson, Inc N Dodge St Iowa City, IA T: F: M: E: Rich.Young@Pearson.com RE: Request for Information ADED Standards Based Competency Assessment, Grades December 2013 Dear Ms. Gomez: Starting in the school year, the Arizona Department of Education (ADE) will begin to replace Arizona s Instrument to Measure Standards (AIMS) with next-generation assessments that will provide parents and educators throughout the state with rigorous new measures of student achievement, a higher bar for achieving excellence, and a more unified focus on preparing all students for college and careers. Through these actions, Arizona will take another significant step toward completing the cycle of reform initiated by Governor Brewer and her P- 20 Coordinating Council (now the Arizona Ready Council) more than four years ago. Pearson s history of support for Arizona s assessment programs stretches back to the very beginning of the AIMS program more than 15 years ago, and it has been an honor and a privilege to have played such an important role in Arizona s growth and success since that time. In fact, with more than 650 full-time and 1,500 part-time Arizona-based employees, many of whom are products of (and whose children attend) Arizona schools and postsecondary institutions, we view it as our responsibility to do all we can to help Arizona succeed. Throughout our 90-plus years in educational assessment, we have helped many other states through transitions similar in magnitude and impact as the one Arizona has now undertaken, providing consistent leadership and innovation through technology and advanced measurement research. Achieving the gains Arizona now seeks will require new assessments that reflect the increased rigor of Arizona s standards, leverage technology to provide deeper insight into students knowledge, skills, and abilities, and ultimately serve to illuminate each child s path to college and career readiness. As you prepare your Request for Proposals (RFP) to select Arizona s next assessment system, the ADE will have a wide range of options from which to choose and, as a result, a number of very important decisions to make. While the use of technology in next generation assessments allows for richer, more authentic assessment, computer-based testing presents an entirely new set of practical and logistical challenges for schools, especially in states like Arizona where high-stakes online testing is new. Testing systems must work with schools existing hardware and software environments, be easy for local technology support personnel to set up and manage, and be easy for students and teachers to use. In addition, schools or districts that lack sufficient hardware, network, or staff capacity for full online testing implementation will need the flexibility to continue testing on paper as their capabilities and experience with computer-based testing grow over time.

3 Having implemented high-stakes online testing in thousands of schools and districts in more than a dozen states over the past 13 years, we have the knowledge, experience, and insight it takes to successfully navigate these challenges. Our PearsonAccess and TestNav platforms have been specifically designed to meet the unique requirements of both paper and online testing, offering schools and districts the flexibility to simultaneously manage the demands and meet the requirements of testing in both modes. Options we can deliver or support for Arizona include: The Partnership for Assessment of Readiness for College and Career (PARCC). Schools and districts throughout Arizona will be administering PARCC field tests through PearsonAccess and TestNav in spring These same systems will also be used for the first operational administration in spring ACT Aspire. A new assessment system developed jointly by ACT, Inc. and Pearson to measure growth to college and career readiness from grade 3 through 10 in reading, writing, mathematics, and science. With the optional ACT Assessment serving as its capstone at grade 11, ACT Aspire can also be combined with ACT s Quality Core assessments to provide additional alternatives for measuring readiness through end-ofcourse achievement. Custom Assessments. Using portions of Arizona s existing item bank in combination with next-generation items licensed from Pearson or one of the two national assessment consortia, Arizona could create custom assessment blueprints to incorporate a mix of item types that would be more efficient and less costly to create, administer, and score. These assessments could then be ratcheted up as experience, capabilities, and available resources increase over time. To remain competitive, Arizona students must keep pace with their peers across the country and around the world. To facilitate their growth and progress, Arizona s assessment systems must keep pace as well. Pearson can provide Arizona with the stability it needs to move forward with confidence, and the advanced research and technology it takes to accelerate growth and achievement for all Arizona students. Regardless of the path Arizona chooses, Pearson will be there to help lead the way. I have actively followed the evolution of Arizona s education system for nearly a decade now and have had the privilege of personally supporting the success of your current assessments over the last five years. As Arizona continues to advance its education reforms, I look forward to the opportunity to continue serving the ADE any way I can. If you have any questions about the information we have provided in our enclosed response or need further information of any sort, please do not hesitate to contact me at or via at Rich.Young@pearson.com, and I will be glad to assist you as quickly and efficiently as possible. Thank you for your consideration. Sincerely, Rich Young Vice President, State Services Assessment & Instruction Pearson

4 REQUEST FOR INFORMATION RFI # ADED Description: Standards-Based Competency Assessments, Grades 3-11 Due date: December 5, 2013 Time: 3:00 p.m. (mst) General Information In accordance with ARS , the State Board of Education (SBE) through the Arizona Department of Education (ADE) hereby gives notice via this Request for Information (RFI), of our desire to receive responses, feedback, suggestions and comments regarding the items set forth herein. In accordance with R2-7-G301, responses to the RFI are not offers and cannot be accepted to form a binding contract. To the extent allowed by law, information contained in a response to an RFI may be considered confidential until the procurement process is concluded or two (2) years from when the RFI s due date has elapsed, whichever occurs first. Responders are solely responsible for expenses associated with responding to the RFI. The State is under no obligation to the responders in so far as the next steps to this process are concerned. Responses to the RFI shall be submitted as follows: One (1) copy of the informational documents, in PDF format, shall be submitted to ADE, c/o Chief Procurement Officer, Lupita Gomez; Procurementinbox@azed.gov by December 5, 2013 at 3:00pm, MST. SBE is issuing this Request for Information (RFI) to solicit data from vendors of educational assessments that could be used in school year as standards-based competency assessments for grades Background: A.R.S requires the SBE to adopt and implement a criterion-referenced assessment to measure pupil achievement in the board adopted academic standards in reading, writing and mathematics. In addition to meeting federal testing requirements under the Elementary and Secondary Education Act, this assessment is used for state mandated accountability measures, including Arizona s school accountability system (A.R.S ), teacher and principal evaluations (A.R.S ), and third grade retention (A.R.S ). In June 2010, the SBE adopted new English Language Arts and Mathematics standards designed to demonstrate college and career readiness and aligned with the Common Core Standards. The implementation of these new standards was phased in over several years, and the new standards will be tested for the first time in school year The SBE issues this RFI in order to better understand the assessment options that may be used as a statewide standards-based competency assessment for grades 3-11 starting in SBE welcomes responses for both existing assessments, and those under development that will be ready for implementation in The information gathered will help inform the SBE as it further develops a Request for Proposal (RFP) to procure a final assessment for use in Arizona. Arizona Department of Education Request for Information October 24, 2013 Page 1 of 4

5 Information being requested: This RFI requests that vendors provide brief descriptions and cost estimates for any existing vendor products or systems or any customized products they have developed with state(s) or other entities which meet the criteria of one or more of the components described below. This RFI is being issued to obtain information only and is not intended to result in contracts or vendor agreements with any respondent. Respondents to this RFI will not be excluded from eligibility to participate in any future requests for proposals (RFPs) based on their decision to respond to this RFI or on the content of their responses to this RFI. While we are interested in responses that encompass all functional components, we are also interested in responses that cover discrete functional components of the entire system. Please provide answers to the questions in the prescribed format and supporting materials that describe the proposed assessment and related issues. 1. Background a. Provide a brief history of the organization and its governance structure. b. Identify the individuals from the organization that will be working with Arizona officials on all aspects of the assessments implementation. 2. Overview of assessment a. Describe the assessments, the grades assessed, the subject areas included, and the formative, diagnostic and summative components. Also describe available end-of-course assessments for grades nine through eleven. b. Describe the timeline for the development of the assessments to ensure full implementation by the academic year. c. Describe how, and the degree to which, the assessments are specifically aligned to Arizona s academic standards in mathematics and English language arts (reading and writing), include any alignment studies, if available. d. Describe how the assessments results can be compared to other states criterion-referenced assessments expected to be in use beginning in the academic year. e. Describe how the assessments are aligned to college / career expectations. Describe the validation process, including the role of post-secondary education in establishing the readiness expectations. Include any alignment studies, if available. f. Describe the available accessibility features, as well as assessment accommodations for individuals with disabilities and English Language Learners. g. Describe any practice and/or sample assessments that are available. 3. Computer-based assessments a. If applicable, describe the computer-based option(s) for the administration of the assessments. b. If applicable, describe the technological specifications for the administration of the assessments. This should include specifications for computer hardware, input devices, security requirements, bandwidth, web browser requirements, and platform software. Arizona Department of Education Request for Information October 24, 2013 Page 2 of 4

6 c. Describe any available training and technical support that is available for the computer-based assessments. d. If the assessment is computer-based is there a paper / pencil option? 4. Assessment administration a. Describe the total anticipated testing time for each assessment (mathematics, reading, and writing) by grade level. If computer-based, include the calculated student to device ratio. b. Identify the anticipated testing window for each assessment by grade level. c. Describe the training needs and available training for teachers and administrators pertaining to the administration of the assessments. 5. Assessment standards setting and scoring a. Describe the standards setting process. b. If already established, describe the performance levels and the performance level descriptors for each category. c. Describe the score reports available to teachers, students and parents. The description should include: i. How the reports illustrate a student s progress on the continuum toward college and career readiness, grade by grade, and course by course; and ii. How the reports are instructionally valuable, easy to understand by all audiences, and are delivered in time to provide useful, actionable data to students, parents, and teachers. d. Describe the process and timelines for scoring the assessments. Include computer-based and pencil / paper processes and timelines, as applicable. e. Describe how scores on the assessments will be comparable to other common college/career ready assessments. 6. Assessment development a. Describe how the development of the assessments will adhere to the principles of universal design, so that the testing interface, whether paper- or technology-based, does not impede student performance. b. Describe any comparability studies between the paper/pencil and computer-based assessments. c. Describe the processes for item development. The description should include: i. How the reading and writing items will require students to demonstrate a range of higher-order, analytical thinking and performance skills in reading, writing and research based on the depth and complexity of the standards, allowing robust information to be gathered for students with varied levels of achievement; and ii. How the mathematics items will require students to demonstrate a range of performance based on the depth and complexity of the standards, allowing robust information to be gathered for students with varied levels of achievement. d. Describe the procedures used to ensure all test items are properly aligned to applicable standards and avoid bias. Include the role of state representatives in these processes. Arizona Department of Education Request for Information October 24, 2013 Page 3 of 4

7

8 Table of Contents Executive Summary... ES 1 Background Overview of Assessment Computer-Based Assessments Assessment Administration Assessment Standards Setting and Scoring Assessment Development Assessment Cost Table of Contents TOC 1

9 Executive Summary Since the origins of Arizona s first Race to the Top application in 2009, Governor Brewer s education reform agenda has consistently focused on college and career readiness as the ultimate standard by which success for all Arizona students should be defined and measured. By establishing rigorous, highquality academic standards and assessments as one of Arizona s four pillars of reform, Governor Brewer has established high expectations for achievement that educators throughout the state are working hard to fulfill. Unified in their support for Governor Brewer s reform agenda, Superintendent Huppenthal, the Arizona Department of Education (ADE), and the Arizona State Board of Education have undertaken a series of bold actions to implement some of the most fundamental changes to Arizona s education landscape since Arizona s Instrument to Measure Standards (AIMS) was introduced more than 15 years ago. Now, with Request for Information (RFI) ADED for Standards-Based Competency Assessments Grades 3 11, the ADE has entered the final stages of Arizona s transition to a new assessment system, a process that has been more than four years in the making. To inform the ADE s development of an RFP that fulfills Arizona s original reform vision, this RFI seeks important background information on reading, writing, and mathematics assessment options available for implementation in that offer the following high-level features: Alignment with Arizona s College and Career Ready Standards Options to create a more comprehensive and balanced system of formative, interim, diagnostic, endof-course, and summative assessments Options for both paper-pencil and computer-based administration Comparisons to college and career readiness indicators and measures in other states Accessibility and accommodations for students with disabilities and English language learners Reporting options to help parents and educators better understand their students needs and progress toward readiness To better prepare students for success in an increasingly competitive, global economy, states across the nation are implementing innovative new next-generation assessment systems that encompass these key features and more. However, with so many changes and so much riding on Arizona s assessment system, failure, as they say, is not an option, and parents, teachers, administrators, and support staff are understandably anxious about the many unknowns that lie ahead. Fortunately, as the ADE s current contractor for both the AIMS and Arizona s English Language Learner Assessment (AZELLA), Pearson can help deliver or support a variety of future assessment options that Executive Summary ES 1

10 fulfill Arizona s vision for a next-generation, college and career readiness assessment system while providing your stakeholders with a greater sense of comfort, confidence, and stability. These options include: The Partnership for Assessment of Readiness for College and Career (PARCC). Schools and districts throughout Arizona will be administering PARCC field tests in spring 2014 using TestNav, our proven online delivery platform, and PearsonAccess, the same online administrative portal used to manage AIMS and AZELLA. These same systems will also be used for the first operational PARCC administration in spring 2015, providing Arizona with a high degree of familiarity and continuity. Assessments from ACT, Inc. and ACT Aspire. ACT Aspire is a new assessment system developed jointly by ACT, Inc. and Pearson to measure growth to college and career readiness from grade 3 through 10 in English, reading, writing, mathematics, and science. With the optional ACT assessment serving as its capstone at grade 11, ACT Aspire can also be combined with ACT s QualityCore assessments to provide additional alternatives for measuring readiness through end-ofcourse achievement. Custom Assessments. Using portions of Arizona s existing item bank in combination with nextgeneration items licensed from Pearson or either of the two national assessment consortia, Pearson can help Arizona create custom assessment blueprints that incorporate a mix of item types designed to be more efficient and less costly to create, administer, and score. If desired, these assessments could be further ratcheted up to provide increased rigor and depth as experience, capabilities, and available resources increase over time. With each of these options, Pearson offers a common thread that Arizona can rely on, with our proven online technology, large operational capacity, comprehensive program management, and track record of success in supporting Arizona through other high-stakes transitions in the past. Coupled with our experience developing and delivering next-generation college and career readiness assessments in other states like Texas, New York, Kentucky, and Virginia, the ADE can count on receiving superior service and support from Pearson regardless of the role we play. Our Support for PARCC As a governing state in the PARCC consortium, Arizona is well familiar with the features and benefits of the PARCC assessments. Pearson is strongly committed to PARCC s success and is currently supporting the consortium through prime contracts for the Technology Readiness Tool, item development, and online test delivery and scoring. Arizona schools and districts participating in the spring 2014 paper and online field tests will have an opportunity to experience firsthand the PARCC assessments and Pearson s delivery support for PARCC, with many schools participating in online testing for the first time. If Arizona should choose to continue with the PARCC assessments in , Pearson will be on board to support a smooth transition and provide continuity to Arizona districts and schools for online test delivery. In addition, we continue to pursue opportunities that would expand our service to PARCC beyond online delivery in spring However, due to the ADE s potential role in PARCC procurements, it would be inappropriate for us to further elaborate on the PARCC assessments or speculate on the potential scope of Pearson s future support for PARCC in this response. Executive Summary ES 2

11 Teaming with ACT, Inc. and ACT Aspire The digital revolution is well underway in Arizona classrooms, so to achieve the kind of gains Arizona now seeks, the ADE needs an assessment system designed to support and promote the changes in instruction now taking place. With college and career readiness as Arizona s ultimate goal, no other company is as qualified to provide Arizona s next assessment system as ACT, whose flagship ACT assessment has served as the gold standard for college entrance nationwide for more than 50 years. Throughout its history, ACT s research-based programs and services for educational assessment, workforce development, career planning, and professional certification and licensure have become mainstays of the American educational and business landscapes. Like Pearson, ACT is deeply involved in supporting Arizona s education system, through innovative programs like the Arizona ACT Initiative sponsored by the Governor s Office of Education Innovation and Helios Education Foundation, and the Gaining Early Awareness and Readiness for Undergraduate Program (GEAR UP), a college access program offered through Northern Arizona University designed to significantly increase the number of students from low-income communities who stay in school and are prepared to succeed in postsecondary education. 1 With common origins at the University of Iowa dating back to the 1950 s, Pearson and ACT have enjoyed more than a half-century of collaboration on ACT programs, including ACT Explore, ACT Plan, ACT QualityCore, the Graduate Management Admissions Test (GMAT), and support of ACT administration as part of 13 statewide testing programs. Through our most recent collaboration, ACT and Pearson have formed a new entity, ACT Aspire, LLC, to create ACT Aspire, a next-generation assessment designed to measure growth toward college and career readiness for students in grades 3 10, with Work-Ready and ACT Composite prediction scores for students in grades 9 and 10. Launching in spring 2014, the ACT Aspire assessment system includes summative, diagnostic, and classroom-based assessments that can be used in combination with The ACT as its capstone for grade 11 and with ACT QualityCore assessments for measuring end-of-course achievement. With technology and operations support provided largely by Pearson, each of these products will be available for both paper and online administration by the spring 2015 administration. Options to Create a Custom Assessment If Arizona should determine that a shared or pre-built assessment will not fully meet the state s needs, the ADE could procure a custom assessment built to a blueprint of Arizona s own design, using a combination of high-quality, well-aligned AIMS items and items licensed from PARCC, the SMARTER Balanced Assessment Consortium, and/or Pearson. The benefits of this approach would include strong alignment with Arizona s College and Career Ready Standards and a greater ability to control content selection, cost, testing time, accommodations, reporting, and compatibility with Arizona s emerging Ed-Fi and Common Education Data Standards architecture. 1 Citation: Arizona Board of Regents. Northern Arizona University, Flagstaff, Arizona Executive Summary ES 3

12 Depending on the source(s) of items selected, the ADE and Pearson, working collaboratively together as we have in the past, would likely be able to assemble and deliver custom operational paper and online forms by spring In addition, there would be a strong likelihood that custom assessments built from these sources would be able to provide equivalent measures for comparability to other states results. Again, with solid operational support and proven technology from Pearson, this approach would provide for reduced risk compared other custom options and a smooth transition for all Arizona stakeholders. Proven Delivery, Powered by Pearson In addition to the significant curricular and instructional changes taking place throughout the state, one of the biggest challenges Arizona faces is the transition to online testing. Over the past decade, Pearson has implemented more than 20 online assessment programs for state and national customers, including current customers in Colorado, Florida, Georgia, Maryland, Mississippi, Tennessee, Texas, and Virginia. Our proven technology and ability to successfully manage states transitions from paper to online testing, coupled with our experience and established relationships throughout Arizona, make us well qualified to help lead your program into the future of next generation assessment. When evaluating an online test delivery system, Arizona should consider five basic criteria to determine which system is the right fit for its assessment program: Security to protect test content and prevent cheating Consistency for students to receive a fair and comparable testing experience, regardless of the hardware, software, or device they use Scalability to complete statewide testing efficiently and on time Reliability to protect students from disruptions and the potential for lost work, allowing them to focus on performing to the best of their abilities Total Cost of Ownership to reduce the local burden of preparing for and supporting testing through zero footprint, industry-standard software solutions With tens of millions of high-stakes online assessments delivered over the last decade, more than 11 million so far this year, Pearson s TestNav and PearsonAccess online platforms offer Arizona a proven solution that fulfills all of the above requirements. For each of the assessment options detailed throughout our response, Pearson provides online delivery, operational capacity, and program management support in subcontractor, partnership, and prime contractor roles, affording ADE the comfort of knowing that no matter which route you choose, your trusted ally can continue to provide Arizona with strong support and dependable service. Additional Considerations As you prepare your RFP, we would like to offer some additional considerations based on our experience in other states around the country. One such consideration is to create or maintain a more cohesive overall assessment system that encompasses all subject areas and even other types of assessments. For example, other states undergoing similar transitions are requiring that new assessments for core subjects in English language arts (ELA) and mathematics continue to be closely coordinated with legacy programs Executive Summary ES 4

13 as well as new or existing assessments in science or other subject areas. By taking a more holistic, comprehensive approach to reform, the ADE can help educators make significant advancements in foundational skills while continuing to support the importance of other subject areas. This type of integrated approach can help promote greater cross-curricular literacy, a key tenet of Arizona s new ELA standards. Maintaining a more cohesive and unified assessment system also helps reduce the overall administrative burden on schools, making it less disruptive to introduce other incremental reforms over time. Another potential consideration pertains to requirements for integration with state data systems. Given the ADE s ongoing initiative to implement the Arizona Education Learning and Accountability System (AELAS) and Arizona Education Data-driven Decision System (AzED 3 S), your RFP for a new assessment system could include specifications for operational synergies that would likely be more costly and timeconsuming to add later. Thank You With deep roots in Arizona s public education system and more than 2,000 full- and part-time Arizonabased employees, Pearson is deeply invested in your success. Regardless of what the future may bring, as citizens of Arizona s business and education community, we view it as our responsibility to do everything within our power to see that all Arizona children have a chance to succeed on their path to college and career readiness. We pledge our support to this important undertaking, and we look forward to continuing our successful relationship with you. Thank you for providing us with this opportunity. Executive Summary ES 5

14 1. Background Requirement a. Provide a brief history of the organization and its governance structure. Response About Pearson As the leading education services company, Pearson is serious about evolving how the world learns. We apply our deep education experience and research, invest in innovative technologies, and promote collaboration throughout the education ecosystem. Real change is our commitment and its results are delivered by connecting capabilities to create actionable, scalable solutions that improve access, affordability, and achievement. Pearson currently provides large-scale assessment services in more than half of US states as well as for the US Department of Education, the College Board, the National Board for Professional Teaching Standards, and for the Partnership for the Assessment of College and Career Readiness (PARCC). Pearson contracted with the Arizona Department of Education (ADE) in 1998 to create Arizona s Instrument to Measure Standards (AIMS) and helped manage the program s evolution through 2004, in compliance with the No Child Left Behind Act of In 2007, Pearson conducted a special study for the AIMS Grade 8 science assessments, comparing results from four field test forms administered both on paper and online through Pearson s TestNav system. Results showed no statistical difference in student results for tests administered in either mode. In 2013, Pearson collaborated with the ADE to introduce college and career readiness target scores into AIMS score reports. These targets, established for Arizona by the National Center for Educational Achievement (NCEA, a department of ACT, Inc.), serve as grade-level indicators of whether students are on target to graduate from high school ready for college and career. Pearson has continued to serve as the ADE s AIMS contractor since In 2004, the ADE contracted with Pearson to implement Arizona s first statewide English language proficiency assessment, the Stanford English Language Proficiency which was subsequently customized to better align with Arizona s English Language Proficiency (ELP) Standards and renamed the Arizona English Language Learner Assessment (AZELLA). In 2011, Pearson teamed with WestEd to create a third edition of the AZELLA based on Arizona s new ELP Standards, which were revised in partnership with WestEd to align with the academic expectations of Arizona s College and Career Ready Standards. 1. Background 1 1

15 With the new AZELLA, Arizona has successfully satisfied stringent new federal requirements and become the first state in the country to implement standardized, technology-based speaking assessments using artificial intelligence (AI) scoring. A World of Resources from Pearson Pearson s three main lines of business represent three key stages of learning: school, higher education, and professional. These lines of business serve markets around the world with a wide range of highquality products and services in assessment, instruction, technology platforms, and educator services. Based in London, Pearson employs approximately 36,000 people in more than 60 nations around the world, including more than 650 full-time and 1,500 part-time employees in Arizona. Pearson Organizational Chart. The State and National Services groups under Assessment & Information are part of a suite of education companies comprising Pearson Education. Douglas Kubach is President and CEO of Pearson s US Assessment and Instruction (A&I) group, a business unit of NCS Pearson Inc., of Bloomington, MN (incorporated in 1962). NCS Pearson Inc Green Valley Drive Bloomington, Minnesota NCS Pearson Inc. is owned by PN Holdings Inc., which is owned by its ultimate parent, Pearson plc (a publicly held U.K. company), or its subsidiary(ies). 1. Background 1 2

16 Senior Leadership. Pearson s senior corporate leadership team has experience and diverse skills to oversee the success of Arizona s assessment programs. The A&I leadership team represents each key area of our large-scale assessment business, setting high standards for quality, timeliness, and integrity in the measurement, operations, and technology services Pearson delivers for Arizona. Insights for Innovation Expectations for educators and students have never been higher. Students today must be prepared for a dynamic, global economy one that calls on people to use data more thoughtfully, think creatively, and work productively in teams. Pearson s Research & Innovation Network (researchnetwork.pearson.com) works toward achieving these goals in four ways: Inventing new digital technologies to support educators in adapting instruction to individual student needs Translating research into recommendations and services that help educators overcome instructional challenges Collaborating with leading experts and practitioners to conduct research and development Sharing and receiving intellectual insights during conferences, through publications, when collaborating with educational stakeholders, and by consulting with foundations on the innovations needed as education navigates its way from the pre-digital to the digital 1. Background 1 3

17 The Research & Innovation Network comprises five centers with distinct, but related, research agendas including the following: The Center for NextGen Learning and Assessment conducts research on innovative new item types, performance assessments, and scoring methods to help understand students day-to-day progress and accelerate their learning. The Center for Online Learning studies teaching and learning through virtual, blended, adaptive, and other elearning technologies through multiple devices to identify effective practices and recommend strategies for personalized, digital instruction at scale. The Center for Digital Data, Analytics, and Adaptive Learning explores new ways to use the massive amounts of data generated by online learning, gaming, and other educational technologies to assess, enable, and personalize learning without the disruption of traditional tests. The Center for College and Career Success helps develop new ways to define, measure, and track students readiness to enter college or the globally competitive workforce. The Center for Educator Effectiveness investigates and promotes new methods to measure teachers performance, develop their full potential, and achieve greater educational outcomes for their students. Each center is led by accomplished researchers who both provide intellectual leadership in the field and advise practitioners working at the front lines of education. Their work constantly informs and enhances the services we offer and provide our customers with additional opportunities for collaboration to advance their education reform programs. 1. Background 1 4

18 A History of Advancing Educational Assessment As the nation s most comprehensive provider of educational assessment products, services, and solutions, Pearson has been a trusted collaborator in district, state, and national assessments for 90 years. We help students, adults, educators, and professionals use assessment, research, and innovative technologies to promote learning and personal development while advancing academic achievement. Pearson has been a trusted collaborator in district, state, and national assessments for 90 years. New technologies have increased collaboration within and among states just as economic pressures to remain globally competitive drive efforts to make learning more effective, accessible, and personalized. In this dynamic environment, the challenge is to design comprehensive, new assessment systems that accelerate educators ability to personalize learning for every student. Leading the Way in Next Generation Assessment Innovative approaches to item development and scoring, such as the expanded use of performance tasks, computer adaptive testing, and automated scoring, are allowing us to more effectively measure what students know and can do. These next-generation assessments move beyond snapshots of student achievement to capture complex performance and track student growth over time, leveraging new digital technologies to provide richer, more timely feedback to teachers, students, parents, and policymakers. Expert Test Design, Development, and Measurement Services At Pearson, test development takes place within our Test, Measurement, and Research Services (TMRS) group rather than across several departments. Having our content experts and psychometricians in the same department facilitates communication, project planning, and close collaboration for the most efficient approach to test development. Our established processes allow our staff to develop items or work with educators to develop items that align to applicable content standards, adhere to the principles of Universal Design, and reflect best instructional practices. At Pearson, our content development and psychometric research capabilities come together to provide the following: 1. Background 1 5

19 Pearson Content Development and Psychometric Research Capabilities Multiple-choice, constructed response, and technology-based innovative item development Test blueprints Statistical analyses Item development aligned to state standards Item reviews for content, bias, and universal design Test form development, including forms adapted for online administration Student/device interaction, usability and comparability studies Automated item banking, tracking, and test forms construction Calibration, scaling, and equating using a broad array of classical and IRT-based measurement models Special studies to examine various assessment issues (e.g., comparability studies, generalizability analyses, factor analytic studies, cognitive labs) Question and Test Interoperability (QTI) and Accessible Portable Item Protocol (APIP) standards Pilot, field test, and operational test design, sampling, and data analysis Validity and reliability investigations Differential Item Analysis (DIF) and other statistical data analyses, such as item and scaling drift Standard setting Accommodated testing of English as a second language populations and examinees with disabilities Performance and automated scoring studies Preparation of technical documentation and reports Presentations of technical data in a variety of media and to various audiences, including state boards of education and state legislators Support for states as they defend and explain how they are meeting requirements of Federal Peer Review and measurement best practices Developing State Assessments. Pearson psychometricians and content development specialists support our state customers through use of sound methodologies in test development. Dr. Jon S. Twing, Pearson s Chief Measurement Officer, directs three functional teams: Measurement Services, led by Dr. Walter Denny Way Research and Innovation Network, led by Kimberly O Malley Learning Integration, led by Dean Brown Measurement Services Arizona s College and Career Ready Standards provide guidance about what students should understand and be able to do at critical points in their education.developing tests that fairly and reliably measure student learning for these new standards takes experience and capacity hallmarks of Pearson capabilities. Our item development work for the PARCC consortium has added greatly to our experience, insight, and ability to develop high-quality, innovative new items aligned with college and career ready expectations, offering Arizona a unique advantage. To support item and test development and test administration in a changing educational landscape, Pearson provides customers with the content and psychometric support shown in the following figure. 1. Background 1 6

20 Measurement Services for Pearson State Customers Initial evaluation of test items and prompts, including alignments Content development in all major subject areas, including general population, English language learners, and special needs students Incorporation of stimuli, items, and prompts into pilot and field test forms Participation in internal and external item/prompt content, bias, and data review Item/prompt scoring and subsequent analyses of pilot test data Test form development, including descriptions of test blueprints and alignment to clients learning standards, practice forms, and descriptions of their contents Accommodated versions of an assessment Quality control activities to maintain consistency across item/test form development, administration, scoring, and reporting functions Development, maintenance, and support for test security measures Securing of permissions, payment on behalf of clients, tracking, and records handoff Descriptions of test administration procedures Training and other consultation Supporting Assessment Quality. Our established processes enable our staff to develop items that align to client content standards, adhere to the principles of Universal Design, and reflect best instructional practices. To give each student the best possible opportunity to demonstrate his or her full range of knowledge and skills requires a full range of Pearson personnel including artists, content specialists, researchers, copyeditors, and fact checkers. Many of our staff hold teaching certifications and have experience teaching in K 12 classrooms. Their commitment to quality education shows in the care they give to creation, review, and production of every test item. This care results also from established processes developed over decades of assessment experience. Our procedures enable Pearson content specialists to develop items that align to client content standards, adhere to the principles of Universal Design, and reflect best practices, including the 1999 AERA, APA, and NCME Standards for Educational and Psychological Testing. Applicable APA Standards guide test development. Psychometric Services Sound methodologies and continuing research lead to assessment results that students, parents, educators, and stakeholders can trust. Pearson psychometricians, statisticians, and research personnel use sound methodologies in technical areas of test development. We are careful to consider the content, format, and context in which tests are given, as well as the potential consequences of using the test to make judgments about students and instructional practices. Many of our psychometricians are nationally recognized experts in high-stakes standards-based assessment. They participate in review and publication of peer research, and they have been consulted on revisions to the APA standards. Dr. Walter Denny Way, Senior Vice President of Psychometric and Research Services, served as a member of the joint committee that revised the AERA, APA, and NCME Standards for Educational and Psychological Testing. The APA Standards form the backbone for sound psychometric practice in educational and psychological testing, promote the sound and ethical use of tests, and provide a basis for evaluating the quality of testing practices. 1. Background 1 7