Performance Management Framework Guidelines and Technical Guide

Transcription

1 Performance Management Framework Guidelines and Technical Guide November 2011

2 About the DC Public Charter School Board The DC Public Charter School Board (PCSB) currently oversees 53 schools on 99 campuses, which serve nearly 30,000 students from every ward of the city. The organization s mission is to provide high quality public school options for District of Columbia students, families, and community through four functions: A comprehensive review application process ensures that the PCSB only approves charter school applications that will prepare and train students for post secondary experiences and individual career paths; Effective oversight holds schools to high standards for results, with extensive reviews and data collection, and makes oversight decisions with the best interests of students in mind; Meaningful support provides clear feedback and increased oversight to low performing schools, and rewards consistently high-performing schools with more autonomy; and Active engagement of stakeholders solicits community input and strives to be responsive to and transparent with all who are impacted by and impact the PCSB and public charter schools. The PCSB s vision is to lead the transformation of public education in DC and to serve as a national model for charter school authorizing and accountability. At the heart of the organization s core values is the belief that every child is entitled to a high quality education that will enable him or her to leave high school well-prepared for college and career. A mayor-appointed governing board of seven with a professional staff of 24 is responsible for the oversight and management of the organization s mission and vision.

3 TABLE OF CONTENTS PERFORMANCE MANAGEMENT FRAMEWORK OVERVIEW... 4 SECTION 1: ACADEMIC EVALUATION - COMMON ACADEMIC INDICATORS, MEASURES, AND METRICS... 5 Overview of academic framework... 5 Common Indicators... 5 Standard schools serving students within grades Elementary school and Middle school framework... 6 High school framework... 6 Summary of academic evaluation frameworks - common academic indicators and measures... 7 Common Metrics... 9 Performance bar metrics... 9 Missing data SECTION 2: ACADEMIC EVALUATION FOR NON-STANDARD SCHOOLS AND PROGRAMS Non-standard schools Early childhood and elementary schools under third grade (PS-2) framework Elementary schools under fourth grade (K-3) Adult education Special education New schools SECTION 3: MONITORING CHARTER SCHOOLS ACADEMIC PERFORMANCE PMF Performance tiers PCSB responses Tier I Tier II Tier III Candidates for Charter Revocation SECTION 4: STRUCTURE OF NON-ACADEMIC EVALUATION FRAMEWORK - COMPLIANCE Overview of framework Compliance SECTION 5: STRUCTURE OF NON-ACADEMIC EVALUATION FRAMEWORK - FINANCE Overview of framework Reporting Assessing Fiscal Management APPENDIX Median Growth Percentile Overview Accountability Plan Guidance Compliance Review TECHNICAL GUIDE Glossary of Measures Metrics and data sources for academic performance measures... 28

4 Performance Management Framework Overview The School Reform Act ( SRA ) grants the D.C. Public Charter School Board authority to hold D.C. public charter schools accountable for fulfilling their duties and obligations under the Act. The PCSB has developed these Guidelines to outline the process by which it will evaluate the performance of the charter schools, including how the PCSB will ensure that each school complies with its charter agreement and applicable law and how the PCSB will track the progress of each school in meeting its student academic achievement expectations. The PCSB s performance management framework is divided into the academic elements of school performance directly related to student outcomes and the non-academic elements of school performance, which include finance and compliance. Each year, pursuant to (a)(3) of the SRA, the PCSB will evaluate the academic performance of each public charter school. The overall assessment will determine whether a school is designated a high-performer, mid-performer, or low-performer. Closer scrutiny will be directed to those charter schools that are failing to meet the goals and student academic achievement expectations set forth in their charters. Schools that are persistently or significantly low performers may be candidates for charter revocation pursuant to the PCSB s authority under (a) of the SRA. The framework also includes an evaluation of a school s performance in non-academic areas, such as financial health and compliance. Each year, pursuant to (a)(3) of the SRA, each school will receive an evaluation of its performance in these areas. As a result of these reviews, schools that show signs of fiscal mismanagement and/or have violated applicable law or the terms of their charter agreement may receive closer scrutiny, including the possibility of recommendations to improve their performance. Schools that continually violate material terms of their charter agreement or applicable law, including violations relating to the education of children with disabilities, may be candidates for charter revocation pursuant to the PCSB s authority under (a) of the SRA. Schools that engage in a pattern of non-adherence to generally accepted accounting principles, engage in a pattern of fiscal mismanagement, or are no longer economically viable will be candidates for charter revocation pursuant to the PCSB s authority under (b) of the SRA. 4

5 Section 1 Academic evaluation - common academic indicators, measures, and metrics Overview of academic framework To assess a charter school s academic performance, the PCSB has developed an academic evaluation framework comprised of indicators, measures, and metrics. This structure has been adapted from a report by the National Consensus Panel on Charter School Academic Quality. 1 Indicators are defined as general dimensions of academic quality or achievement. Measures are defined as general instruments or means to assess performance in each area defined by an indicator. Metrics are defined as the calculation method or formula for a given measure. As an example, a common indicator of student performance is academic achievement level, a common measure of achievement is performance on statewide assessments like the DC-CAS, and a related performance metric is the percentage of students in a school who score at least Proficient on the assessment. This section of the Guidelines focuses on indicators, measures, and metrics common across all schools within each grade-span. Common Indicators For the school year the PCSB used four indicators to measure academic performance: (1) student progress, (2) student achievement, (3) gateway measures, and (4) leading indicators. Figure 1. Academic indicators Student Progress Student Achievement Gateway Measures Leading Indicators The four core indicators of academic achievement apply to every school; however, different weights will be assigned to the indicators depending on the grade span of the school. 1 See A Framework for Academic Quality, National Consensus Panel on Charter School Academic Quality, June 2008, available at 5

6 Standard schools serving students within grades 3-12 Elementary and middle school (3-8) framework Figure 2. Elementary and middle schools Student Progress Student Achievement Gateway Measures Leading Indicators 40% 25% 15% 20% The elementary school framework applies to schools with all or part of the 3-5 grade-span and that include both 3rd and 4th grades. Schools that have only fifth grade in the elementary school span and that span into the middle school range will be considered part of the middle school framework. The middle school framework applies to schools with all or part of the 6-8 grade-span and that include both 6th and 7th grades. Schools where 8th grade is the only middle school year and with all or part of the 9-12 grade-span are included in the high school scoring framework. The elementary and middle school frameworks are designed to place an additional emphasis on student progress and student achievement. Gateway measures are designed to capture key subject area mastery, literacy at the elementary school level, and mathematics at the middle school level. High school (9-12) framework Figure 3. High schools Student Progress Student Achievement Gateway/Post Secondary Measures Leading Indicators 15% 30% 30% 25% This framework applies to all schools with any grades in the 9th through 12th grade-span. The high school framework places less emphasis on student progress. This is largely due to the fact that this indicator is comprised of a measure based on the DC-CAS, and at the high school level, only 10th grade students are tested. The balance is more heavily weighted on achievement than progress, which reflects the fact that, at the high school level, students have neared the end of their public school tenure and the rate of progress is less important than the overall level that has been reached. Similarly, the high school framework places more emphasis on gateway and post-secondary measures that are indicators of overall preparation for college and work-force readiness. For a list of all common measures and metrics please see the Glossary of Measures in the Technical Guide. 6

7 Summary of academic evaluation frameworks - common academic indicators and measures Table 1 Elementary and Middle School Framework Student Progress Median Growth Percentile 1 Reading 20% Median Growth Percentile 1 Math 20% Student Achievement 2 DC-CAS Proficient + Advanced Reading 10% 3rd-5th grades 5% 3 6th-8th grades 5% 3 DC-CAS Proficient + Advanced Math 10% 3rd-5th grades 5% 3 6th-8th grades 5% 3 DC-CAS Advanced Reading 2.5% 3rd-5th grades 1.25% 3 6th-8th grades 1.25% 3 DC-CAS Advanced Math 2.5% 3rd-5th grades 1.25% 3 6th-8th grades 1.25% 3 Gateway 4 DC-CAS 3rd grade reading proficiency 7.5% DC-CAS 8th grade math proficiency 7.5% Leading Indicators Attendance 10% Re-enrollment 10% 40% 25% 15% 20% 1 For more information about Median Growth Percentile please see Appendix A. 2 Elementary and middle grades will be shown separately if at least two grades within the 3 rd -5 th grade range and the 6 th -8 th grade range are represented. If a school does not have two grades within both ranges, the available grades will be combined, i.e., a school with 3-7 th grade would include both 3-5 th and 6-7 th separately; a school with 3-6 th would combine 3-6 th into one score. 3 If elementary and middle grades are shown separately the points available will also be separated so that schools can earn up to 5% of the total points for reaching proficiency for each subject and grade range and up to 1.25% of the total points for advanced. If a school has ES/MS combined, the maximum points for proficiency in each subject will be 10% of the total points for proficiency and 2.5% of the total points for advanced. 4 If a school has both a 3 rd and 8 th grade that school will have two gateways, each worth 7.5% of the total points. If a school has only 3 rd or 8 th that school will have one gateway worth 15% of the total points. 7

8 Table 2 High School Framework Student Progress Median Growth Percentile Reading 7.5% Median Growth Percentile Math 7.5% Student Achievement DC-CAS Proficient + Advanced Reading 10% DC-CAS Proficient + Advanced Math 10% DC-CAS Advanced Reading 2.5% DC-CAS Advanced Math 2.5% Advanced Placement or International Baccalaureate 5% Gateway Graduation Rate 7.5% PSAT (11th grade performance) 7.5% SAT/ACT (12th grade performance) 7.5% College Acceptance rate 7.5% Leading Indicators Attendance 10% Re-enrollment 10% 9th grade credits on track 5% 15% 30% 30% 25% If a charter school does not contain the grade for which a common measure applies, the points associated with that measure will be removed and the total possible points available will be adjusted. For example, a school that terminates in the 11th grade will not have graduation, SAT, AP, or College acceptance data. In this case, the school will be evaluated against the remaining 72.5 points. 8

9 Common Metrics To assess charter schools progress in meeting their goals and academic achievement expectations, the PCSB will use various metrics to assign points to each common measure. The overall percentage of possible points each school receives will determine whether a school is designated as a high-performer, mid-performer, or low-performer. Closer scrutiny will be directed to those charter schools that are failing to meet or in danger of failing to meet the goals and student academic achievement expectations set forth in their charters. Schools that are persistently low performers may be candidates for charter revocation pursuant to the PCSB s authority under (a) of the SRA. Performance bar metrics Most common measures are based on the percentage of students that meet or exceed a certain performance bar. For example, with DC-CAS proficiency, schools are scored based on the percentage of students that meet or exceed the performance bar of Proficient on the DC-CAS. Each student either meets or does not meet the common measure; the metric is based on the percentage of all students that meet the standard. The floor determines the minimum value for which any points will be awarded. Charter schools will receive no points for values that are at or below the floor. For example, the floor for re-enrollment is 55%. A school where 50% of its students re-enroll would not receive any points for the measure even though half of its eligible students re-enrolled in the school. The target determines the value at which the maximum points for a common measure will be awarded. For example, the target for re-enrollment is 90%. A school where 92% of its students re-enroll would receive the full amount of points available for the measure even though all eligible students had not returned to the school. The tables below summarize the performance bar metrics for the common measures. Table 3 Elementary and Middle School Framework Indicator Progress Achievement Gateway PMF Metric Elementary (3-5) Floor Middle (6-8) Floor Target (ES and MS) MGP-Reading MGP-Math DC CAS Reading % proficient + advanced DC CAS Math % proficient +advanced Reading % advanced Math % advanced DC CAS 3rd Reading % proficient DC CAS 8th Math % proficient Leading Indicators Attendance Re-Enrollment

10 Table 4 High School Framework Indicator PMF Metric High School Floor High School Target Progress MGP-Reading MGP-Math DC CAS Reading % proficient + advanced DC CAS Math % proficient + advanced Achievement DC CAS Reading % advanced 0 25 DC CAS Math % advanced 0 25 Advanced Placement and International Baccalaureate 0 15 Gateway Graduation Rate PSAT Performance (11 th grade) SAT/ACT Performance (12th grade) College Acceptance Rate Leading Indicators Attendance Re-Enrollment th grade on track to graduate Missing data Most data required to calculate the score for each charter school s common measures will be collected from third party data sources. The PCSB will rely on charter schools to provide data that are not available through third parties. When data for a common measure are not available due to issues beyond a school s control, such as small sample sizes for growth measures that require statistical calculations, then the points associated with that measure will be removed and the total possible points available will be adjusted. When data for a common measure is missing because a charter school has not provided it to the PCSB in a timely manner, zero points will be awarded for the relevant measure. 10

11 Section 2 Academic Evaluation for Non-Standard Schools and Programs Non-standard schools Non-standard schools include ones that do not fall into the 3-12 grade range or are elementary schools without at least two years of DC-CAS testing data. Non-standard schools will not receive a PMF Annual Performance Review for the or school years and will instead be evaluated with accountability plans. Schools that span the preschool through elementary or middle grades will receive both an accountability plan for any grades served below 3, as well as a PMF for the 3 rd -8 th grades. The accountability plan system is designed to measure and report the academic performance for students at either end of the educational spectrum, while allowing schools the autonomy to identify appropriate measures based on their unique populations. Each school s accountability plan was developed in concert with PCSB staff and the school s leadership. Schools were provided guidance on the minimum and maximum number of targets to include in the accountability plan as well as criteria in the identification of assessments and measures to determine performance (see Accountability Plan Guidance charts in Appendix B). With the exception of schools undergoing charter review, schools and programs using accountability plans were not rated or tiered for the school year. Accountability plan results are determined and reported based on whether the school met or missed its established targets. The accountability plan system is used for the following types of schools and programs. Early childhood and elementary schools under third grade (PS-2) framework Preschools and early years of elementary schools are in the non-standard framework because DC does not mandate a standardized assessment for students in these grades. Schools with both early childhood and elementary school grades but that do not exceed 2nd grade will be considered in the early childhood framework and be evaluated using an accountability plan for the and school years. Elementary schools under fourth grade (PS-3) Schools that end in the third grade are separated from other elementary schools because they do not have at least two years of DC-CAS data to measure student growth in mathematics and reading sufficiently. Adult education Schools offering adult education and G.E.D. coursework are included in the non-standard framework because DC does not mandate a standardized assessment for students in these programs. Special Education For the school year, the PCSB recognizes one charter school as a stand-alone special education entity as 100% o of the school s population is identified with significant physical and cognitive disabilities. All eligible students taking the state-wide assessment at this school are administered the DC 11

12 CAS Alternative Assessment Portfolio to measure their academic and developmental progress and performance. New schools In their first year of operation, charter schools will be held to a modified academic review. Data will be collected on all measures, where available. For schools administering the DC-CAS, student achievement will be measured by the common measures of Proficient and Advanced in reading and mathematics. Beginning in their second year, new charter schools will be held to the same performance framework as existing schools. The growth measure will be based upon two years of data only, and the graduation rate will be modified according to OSSE guidelines. Accountability Plan Guidelines For more information about Accountability Plans please see the PCSB s Accountability Plan Guidelines. 12

13 Section 3 Monitoring Charter Schools Academic Performance Pursuant to the SRA (a), the PCSB has the discretion to revoke the charter of a school that is failing to meet its goals and student academic expectations. Based on the data collected and the metrics applied to a school s academic performance measures as outlined in the previous sections, the PCSB will make an overall assessment of a school s academic performance on an annual basis. The PCSB assesses academic performance using PMF results for standard schools and accountability plan results for non-standard schools. Using the PMF, the PCSB will designate a school as a highperformer, mid-performer, or low-performer. Under the accountability plan system, a non-standard school was not tiered or rated for the school year. Closer scrutiny will be directed to those charter schools that fail to meet student academic achievement expectations set forth in their charters. Standard and non-standard schools that are low performers may be candidates for charter revocation pursuant to the PCSB s authority under the SRA. PMF Performance tiers Using a 100-point scale and based on the scores for the academic scoring screen, standard schools will be identified as Tier I (high-performers), Tier II (mid-performers), or Tier III (low-performers). Tier I schools earn at least 65% of the possible points. Tier II schools earn between 35% and 64% of the possible points. Tier III schools earn less than 35% of the possible points. A school must meet the threshold for points for each tier; points are not rounded up to the next whole number. The threshold points for identifying each tier will be set every few years and identified through an assessment of past overall school performance across all public charter schools. PCSB responses Tier I High performing schools will be publically recognized as such by the PCSB and will be exempt from Program Development Reviews (PDRs) unless conducted as part of the charter review or renewal cycle. Tier II Mid-performing schools will be exempt from PDRs unless conducted as part of the charter review or renewal cycle. Tier III The PCSB will commence a full PDR for low-performing schools during the school year following the academic performance evaluation. The PCSB will suggest that low-performing schools develop a plan to target areas in need of improvement. Charter schools that are low performers may be candidates for charter revocation pursuant to the PCSB s authority under the SRA. Program Development Reviews are conducted at the campus level; multi-campus charter schools will receive a PDR for a Tier III campus, but would not undergo a PDR for a Tier I campus. 13

14 Candidates for Charter Revocation Charter school may become candidates for revocation due to academic or non-academic issues. Under the PMF, candidacy for revocation due to academics will be based on severity, trajectory, or duration of low performance at one or more schools within an LEA. Based on PMF results, standard schools scoring below 20 percentage points in the most recent year, showing 5 percentage point decrease within Tier III from one year to the next, or performing in Tier III for three consecutive years will become candidates for revocation. Under the accountability plan system, non-standard schools that fail to attain the majority of the academic performance goals listed in its accountability plan or come within 90% of all missed academic performance goals on its accountability plan, perform within a minimum of 90% of its accountability plan attendance targets, or maintain sufficient enrollment levels to sustain the economic viability of the school will become candidates for revocation. Any public charter school may be a candidate for revocation of its charter if it commits a violation of applicable law or a material violation of the conditions, terms, standards, or procedures set forth in the charter, including violations relating to the education of children with disabilities. The PCSB retains the right to revoke a school s charter at any time pursuant to its authority under the Act and the terms outlined in of the Act. 1 Poor academic performers are standard schools that 1) score 20% or below on applicable frameworks in the most recent year; 2) decrease in academic score within Tier III by 5 or more percentage points between two consecutive years, or 3) remain in Tier III for three consecutive years. 14

15 Section 4 Structure of non-academic evaluation framework - compliance Overview of framework Schools that continually violate material terms of their charter agreement or applicable law, including the SRA, may be candidates for charter revocation pursuant to the PCSB s authority under (a) of the SRA. The PCSB may use its oversight authority under the SRA (a)(2) to request documents and information from charter schools when it determines it is necessary to carry out its duties under the SRA. Compliance SRA outlines the duties and other requirements of public charter schools. As part of its oversight responsibilities, the PCSB will determine on an annual basis, in coordination with the charter schools, whether sufficient documentation is on file with the PCSB to ensure the charter schools are satisfying these duties and requirements. Further, the PCSB may require schools to provide documents supporting charter schools compliance with applicable law, including the Elementary and Secondary Education Act. These documents include, but are not limited to: Background check reports conducted on employees to ensure the charter school is maintaining the safety of all students attending the school. A report that documents that the charter school s facilities comply with the applicable health and safety laws and regulations of the federal government and the District of Columbia, including the D.C. Fire Prevention Code. The school s informal complaint resolution process and student handbook. Compliance with No Child Left Behind notifications and Highly Qualified Teacher requirements. To ensure charter schools are governed by a Board of Trustees in a manner consistent with the charter granted to them and in accordance with the requirements of the School Reform Act, the PCSB requests charter schools submit board meeting minutes on a quarterly basis and written notification of any board or key personnel changes as these changes occur. PCSB staff will verify that these documents are on file from the previous school year and request a copy of the current Board of Trustees roster. Pursuant to the School Reform Act (a), the PCSB has the duty to ensure public charter schools comply with the annual reporting requirement of SRA (c)(11). The PCSB may use the contents of the annual report to assess a charter school s academic achievement and the extent to which the school is meeting its mission and goals as stated in its charter agreement. Certain events may trigger the PCSB to request additional information from a charter school pursuant to its non-academic evaluation of the school. These events include turnover of key personnel identified in the charter agreement and changes in school enrollment and daily attendance rates as reported in the charter school s annual report. See Appendix C for more a list of documents required for the compliance review. 15

16 Section 5 Structure of non-academic evaluation framework - finance Overview of framework The PCSB must revoke the charter of a school when it determines that the charter school has engaged in a pattern of non-adherence to generally accepted accounting principles or a pattern of fiscal mismanagement or is no longer economically viable. See SRA (b). Accordingly, the PCSB conducts an annual review under the PMF of a charter school s financial performance. Pursuant to this mandate, the PCSB also routinely collects financial statements from schools including interim and audited financial statements and annual budgets. Reporting Charter schools submit financial reports to the PCSB through the Authorizers Oversight Information System (AOIS). The PCSB has established a schedule for the submission of financial reports, and schools initially will be evaluated based on the submission of requested documents by the designated due date. If a charter school does not submit requested financial documents on time and no extension has been granted by the PCSB, the charter school will be subject to a more extensive inquiry by the PCSB. When a public charter school has failed to submit its financial reports in a timely matter, the PCSB may initiate a site-based diagnostic review conducted by a certified public accountant (CPA). The purpose of the review will be to determine the general adequacy of a school s financial management systems and record keeping mechanisms. The emphasis of the review will be on: 1. Verifying budget and cash flow projections and processes are in place. 2. Checking for evidence that the school regularly monitors actual revenues and expenditures against its budget and cash flow projections. 3. Verifying that the school is current in its payment of employee retirement, health insurance costs, and employee withholding taxes. 4. Verifying that there is a long term financial planning process that appropriately considers the impact of expanding facilities and increasing enrollment on cash flow. 5. Verifying that the school s books and records are up to date. 6. Verifying that checking accounts are reconciled in a timely fashion. 7. Verifying that there are appropriate internal controls, particularly with regard to segregation of duties among employees. 8. Verifying that financial reports are provided to the school s Board of Trustees at least quarterly. 9. Verifying that major financial commitments and decisions are approved by the school s Board. 10. Verifying that operating account balances are sufficient to carry the school throughout the year. 11. Verifying that there is a demonstrated technology software package in place to keep accurate financial records and generate reports. 12. Verifying that the accounting system is set up to track revenues and expenditures by funding streams, where required. 13. Verifying that the school maintains proper documentation of grants and donations. Assessing Fiscal Management To track a school s fiscal management, the PCSB uses a General Performance Assessment (GPA). Through the GPA, schools receive a score between 0 (low) and 4 (high) based on eight financial ratios calculated using school-provided financial data. After the ratios have been tabulated, the PCSB will 16

17 apply a weight to each score, sum the products, and calculate the GPA. The school rubric for the eight ratios, and an explanation of each ratio, is contained in the PCSB s Fiscal Policy Handbook. The PCSB Business Oversight Manager will follow-up with school leadership to inquire about a school s unsatisfactory financial performance. Schools may be allowed additional time to comply and provide information to enhance the school s financial picture. If charter schools do not provide adequate supplemental financial data to show that the school is fiscally managed, the PCSB may initiate a sitebased diagnostic review conducted by a PCSB approved auditor or the DC Office of Integrity and Oversight. The purpose of the review will be to determine the general adequacy of a school s financial management systems, record keeping mechanisms, and to ascertain the school s short and long-term sustainability. The emphasis of the review will be the same as the initial diagnostic review performed by a CPA as described above. Schools that engage in a pattern of non-adherence to generally accepted accounting principles, engage in a pattern of fiscal mismanagement, or are no longer economically viable will be candidates for charter revocation pursuant to the PCSB s authority under (b) of the SRA. 17

18 Appendix 18

19 Appendix A: Median Growth Percentile Overview Student Growth Percentiles Individual Student Growth Traditional presentations of students DC CAS scores reflect absolute achievement. These snapshots are useful for describing the performance level of students within a school for any given year but do little to explain the progress students are making. In order to paint a more comprehensive picture of student and school performance, the PCSB, in conjunction with The Office of the State Superintendent, implemented a measure of student progress that compares changes in a student s DC CAS scores to changes made by other students with similar score histories. A student growth percentile measures student progress by comparing one student s progress from year to year to the progress of other students with similar DC CAS score histories. In this way, students are only compared based on their previous DC CAS score, not on demographic metrics (like race, gender, or socioeconomic status). This metric uses scores from all District students (including those at DCPS schools) to determine an academic peer group and calculate individual student growth percentiles. Percentiles are commonly understood values that express the percentage of cases that fall below a certain score. For example: A student with a growth percentile of 77 in 6 th grade mathematics grew as much or more than 77 percent of her academic peers (students with similar score histories) from the 5 th grade DC CAS Mathematics to the 6 th grade DC CAS Mathematics. Only 23 percent of her academic peers grew more in mathematics than she did. or A student with a growth percentile of 34 in 8 th grade reading grew as well or better than 34 percent of her academic peers (students with similar score histories) from the 7 th grade DC CAS reading to the 8 th grade DC CAS reading. This student grew less in reading than 66 percent of her academic peers. Since this growth measure is assessing change in performance, it is possible for students to have performed below the proficiency mark but to have shown a considerable amount of growth. This sort of occurrence could indicate that elements within a school s program are working to help this student advance. Conversely, it is possible for students to be performing well above the proficiency mark but to have shown little growth. In this way, this growth measure serves to put DC CAS scores into greater context. Student Growth Percentiles in the Aggregate Median Growth Percentile To represent the growth of a school, individual student growth percentiles can be aggregated. After significant research and consultation, the PCSB chose the Median Growth Percentile model as the most appropriate measure for reporting the growth of a school. This model finds the median (the middle number for scores arranged in order from least to greatest) of each school s individual student growth percentiles. This is known as the median growth percentile for a school. The DC PCSB will report student growth for DC CAS reading and mathematics for grades 4 through 8, and grade 10. Because the model requires data from at least two grade levels, students in grade 3 (the first testing grade) will not be included in the calculation. 19

20 Appendix B: Accountability Plan Guidance Adult Education/GED Programs Schools using the Adult Education/GED Framework must choose at least 5 targets (ideally no more than 9) from the Academic Indicators. Should a school choose to include more than 9 targets, the total number of targets must be odd. GRADE STUDENT PROGRESS Min/Max Targets Criteria No minimum/ No maximum Can be curriculum based or standardized; PCSB preference is measures be standardized Measures are chosen from what schools already have in place from the school year Targets must assess a cross section of students in programs STUDENT ACHIEVEMENT Min/Max Targets Criteria No minimum/ No maximum Can be curriculum based or standardized; PCSB preference is measures be standardized measures Measures are chosen from what schools already have in place from the school year Targets must assess a cross section of students in programs GATEWAY Min/Max Targets No minimum/ No maximum Optional Criteria LEADING INDICATORS Min/Max Targets No minimum/ No maximum Re-enrollment: Optional Attendance: Optional Criteria 20

21 Early Childhood Programs Schools using the Early Childhood Framework (schools serving preschool and/or prekindergarten students) must choose at least 5 targets and can choose up to 9 from all of the Academic Indicators: Student Progress, Student Achievement, Gateway, Leading Indicators and Mission Specific. STUDENT PROGRESS (PS-2) Min/Max Targets 2 minimum/ 3 maximum Schools can choose a minimum of 2 targets and up to 3 targets Criteria Can be curriculum based or standardized; PCSB preference is K-2 measures be standardized Aligned to Early Learning Standards when possible (PS-PK) Aligned to Common Core Standards when possible (K-2) Measures are chosen from what schools already have in place from the school year Indicators/targets must assess all grades (PS-2). Measures do not have to be the same for all grades STUDENT ACHIEVEMENT (1 st -2 nd grade- K is optional) Min/Max Targets 1 minimum/ 3 maximum Schools can choose a minimum of 1 target and up to 2 targets Criteria Can be curriculum based or standardized; PCSB preference measures be standardized measures Preschool and Prekindergarten students are not included Aligned to Common Core Standards when possible (K-2) Measures are chosen from what schools already have in place for school year Indicators/targets must assess grades 1-2 with K being optional Measures do not have to be the same for each grade LEADING INDICATORS Min/Max targets 1 minimum/ 2 maximum Schools can choose a minimum of 1 target and up to 2 targets Criteria Re-enrollment: Not required for PS-K only schools. Not required, but optional for continuous schools. (i.e., any school that serves post 2nd grade) Attendance: PS-PK (non-compulsory) 88% target K-2 (compulsory) 92% target 21

22 MISSION SPECIFIC (optional for K-2 nd grade) Min/Max Targets No minimum/ 2 maximum Schools can opt not to include a target or to include up to 2 targets Criteria Optional for K-2 Schools can choose the measure but should relate to each gradespan Schools may utilize surveys (e.g., character education surveys, parent satisfaction/involvement) Schools may include an environmental assessment (e.g., CLASS, ECERS, ELLCO) 22

23 Technical Guide Appendix C: Compliance Review INDICATOR DOCUMENTATION LOOK FOR RATIONALE Enrollment of New Students 1. Fair enrollment process Enrollment application; written lottery procedures with dates for enrollment process. Student Suspension and Expulsion 2. Notice and due process Student Health Records 3. Health and safety of students Current year student handbook or other written document that outlines the school's discipline policy and procedures. Option 1: Notice of assigned nurse on staff. Option 2: Copy of staff certificate to administer medications. Lottery cutoff date and date of lottery published in advance (if applicable); waiting list includes name of student, wait list number and current status. Clear explanation of infractions; consequences, rewards, and interventions; clearly outlined due process procedures (including an explanation of the basis for the action). Option 1: Letter from DOH or contracted agency. Option 2: Ensure that there is at least one certificate from school staff who has been certified to administer medications from approved agency Compliance with School Reform Act Section Compliance with School Reform Act Section (g); guidance for PCSB staff when contacted by parents. Compliance with School Reform Act Section (c)(4) and the Student Access to Treatment Act of Background Checks on Employees and Volunteers Current roster of all employees and volunteers (working Documentation includes employee greater than 10 hours at 4. Health and or volunteer name, date of hire, the school) with safety of date of background check, and indication of date students. company that conducted the background check background check. conducted and that a copy of the report is on file. Compliance with School Reform Act Section (c)(4). Employee Handbook 5. Employee handbook or Employment other written document policies and on policies and the procedures governing protection of employment at the confidential school. information. Policies guiding the safeguarding of confidential student data; compliance with relevant employment laws including policies on sexual harassment, equal opportunity hiring, drug-free workplace, and termination. Compliance with School Reform Act Section , FERPA, the Public Education Reform Amendment Act of 2007, and applicable state and federal employment laws. 23

24 Technical Guide INDICATOR DOCUMENTATION LOOK FOR RATIONALE Insurance 6. Appropriate insurance. School Facility 7. Lease/Purcha se Agreement and certificate of occupancy. Certification that appropriate levels of insurance have been secured. Certificate of occupancy. Lease/Purchase Agreement. General Liability, Directors and Officers Liability, Umbrella Coverage, Property/Lease Insurance, Auto Liability Insurance, Workers Compensation. Name and current address listed on the certificate of occupancy; enrollment projections for school year within maximum capacity in the certificate of occupancy. Name and current address listed on the lease/purchase agreement. Compliance with School Reform Act Section (b)(4). Compliance with School Reform Act Section (b)(4)- a Certificate of Occupancy is required at opening and for relocation to a new facility. Compliance with School Reform Act Section (b)(4)- lease or purchase agreement is required at opening, for relocation to a new facility, and for amendments to a lease once it expires. No Child Left Behind (NCLB) 8. School quality and choice. 9. High quality teachers. Board of Trustees 10. Composition. 11. Fiduciary Duty. Communication with parents on school's compliance with NCLB before September 1 or within 14 days of school AYP results. For Title I schools, current year teacher roster with grade and subject(s) taught, HQ status, and how the status was met (HOUSSE, Praxis, Degree, License/Certificate); action plans for all non- HQT staff. Board roster with names and titles. Board meeting minutes. Communication, where applicable, on AYP results, School Improvement Status, school choice options, teacher qualifications. Appropriate documentation for verification of HQT compliance; plan of action to assist teachers in meeting the requirements; appropriate staffing changes to ensure compliance with the law. Odd number of members greater than 3 but fewer than 15; majority DC residents; at least 2 parents of currently enrolled students. Decisions made by the Board are consistent with the charter granted to the school, the School Reform Act, and applicable law. Compliance with NCLB and ESEA guidance. Compliance with NCLB and ESEA guidance to ensure that all elementary and secondary subject area teachers are highly qualified. Compliance with School Reform Act Section and the school s articles of incorporation and bylaws. Compliance with School Reform Act Section

25 Technical Guide Performance Management Framework Technical Guide 25

26 Technical Guide Glossary of Measures Listed below are brief descriptions of some of the common academic scoring measures the PCSB uses to track charter schools progress in meeting their goals and academic achievement expectations. Student Growth Measures Student progress Appears for 3rd -5th grades, 6th -8th grades, and High Schools Median growth percentiles for mathematics and reading, a growth measure based on individual student DC-CAS performance. See Appendix A for a discussion of these growth measures. See documentation provided by OSSE and AIR for Technical Report and Business Rules. Student Achievement Measures DC-CAS Proficient & Advanced Appears for 3rd -5th grades, 6th -8th grades, and High Schools Percent of students scoring at least Proficient and those scoring Advanced on the DC-CAS for math and reading; both Proficient and Advanced measures are included for all grades in which DC-CAS testing occurs. Advanced Placement Appears for High Schools Ratio of number of passing scores for the entire population to number of seniors, written as a percentage. Equivalent assessments, such as the International Baccalaureate exam, may also be substituted. Gateway Measures DC-CAS 3rd grade reading Appears for 3rd -5th grades Percent of students scoring at least Proficient in third grade reading on the DC-CAS DC-CAS 8th grade math Proficient Appears for 6th -8th grades Percent of students scoring at least Proficient in eighth grade mathematics on the DC-CAS Graduation rate Appears for High Schools Graduation rate based on the incoming 9th grade cohort, as defined by OSSE state-wide graduation standards. PSAT 11th grade performance Appears High Schools Percentage of 11th grade students scoring at a level on-track for college readiness in the combined mathematics and verbal sections of the PSAT, defined to be a score of 80 or higher. SAT 12th grade performance Appears for 3rd High Schools Percentage of 12th grade students scoring at a level on-track for college readiness on the combined math and verbal sections of the SAT, defined to be a combined score of 800 or higher. 26

27 Technical Guide College Acceptance Appears for High Schools Percentage of seniors accepted to a college or university. Leading Indicators Attendance Appears for 3rd -5th grades, 6th -8th grades, and High Schools Ratio for the entire population, written as a percentage, of days present (inclusive of excused absences) to days enrolled. Re-enrollment Appears for 3rd -5th grades, 6th -8th grades, and High Schools Percentage of those eligible students who re-enroll. 9th grade credits Appears for High Schools Percentage of 9th grade students on-track to graduate in 4 years, based on OSSE guidelines for completed Carnegie Units and required courses. 27

28 Technical Guide Metrics and data sources for academic performance measures Overview The table below provides details about each of the measures included in the academic performance framework. Information is arranged as follows: Indicator identifies the academic indicator (progress, achievement, gateway, leading) to which each measure belongs Measure provides measure name and indicates which scoring frameworks include it Description provides brief summary of measure Components, method, and additional notes - provides a more detailed description of all steps required to translate raw data into a measure calculation Rationale available rationale for why the metric has been designed as it has Unless otherwise indicated, all measures are grouped by the grade span of the performance framework to which they apply. Grade-spans are as follows: ES/MS: 3-8 HS: 9-12 Data sources and methodology Indicator Measure Description Components, method, additional notes Rationale Student Progress Student Achievement Median Growth Percentile (ES/MS and HS) DC-CAS Proficient in math and reading (ES/MS and HS) Growth measure which tells how much change or growth there has been in performance from year to year. % of students scoring Proficient or above on the DC-CAS Components Median for 3rd-8th(or all grades within that range the school serves) grade student growth percentiles in reading Median for 3rd-8 th grades (or all grades within that range the school serves) student growth percentile in Mathematics References: h%20%25%20.pdf Colorado_Growth_Model.pdf For more information see the MGP Business Rules produced by OSSE and AIR. Components DC-CAS scores by grade for the most recent year Number of students tested for the most recent year Method A measure of student progress that compares changes in a student s DC CAS scores to other students with similar score performance profiles. It tells how much change or growth there has been in performance from year to year. Requirement of two years of DCCAS results of each student. Aligns with the AYP calculation rules, in which students in the Fully Academic Year status and ELL students residing in the US for more than 12 months are included. 28

29 Technical Guide Indicator Measure Description Components, method, additional notes Rationale Total # of Math proficient student in that SY *100 Total # tested OR Total # of Reading proficient student in that SY *100 Total # tested Student Achievement DC-CAS Advanced in math and reading (ES/MS and HS) % of students scoring Advanced on the DC-CAS Components DC-CAS scores by grade for the most recent year Number of students tested for the most recent year Method Calculated as with DC-CAS proficient Total # of Reading Advancedstudent in that SY Total # tested *100 OR Total # of Math Advancedstudent in that SY *100 Total # tested Aligns with the AYP calculation rules, in which students in the Fully Academic Year status and ELL students residing in the US for more than 12 months are included. Student Achievement Advanced Placement performance (HS) Number of passing exams per th grade students Components # of passing AP or IB exams in the current school year 12 th grade enrollment in the most recent year Method # of passing AP exams and IB exams) *100 # of 12th grade students l # of reading proficientin the 3rd grade Total # of 12th graders tested *100 Additional notes Passing exams are all those scoring 3 or above for AP and 4 or above for IB in the current school year One student may contribute multiple passing exams 9-11 grade students may also contribute passing exams Any HS with the 12 th grade that does not offer either AP or IB All passing exams contribute to the score, regardless of who takes the test Expressing passing exams as a per 100 score adjusts to reflect size of school and corresponds more closely to national reporting norms Using only 12 th grade as the denominator ensures all schools are counted equally regardless of what grades they offer Using full 12 th grade class as denominator enables measure to track passing scores without creating incentives for 29

30 Technical Guide Indicator Measure Description Components, method, additional notes Rationale exams will be given 0 points on this metric. schools to limit size of AP classes Gateway DC-CAS Proficient % of 3 rd grade Components Critical gateway year for in 3 rd grade reading students scoring at least Proficient 3 rd grade DC-CAS scores Method childhood literacy (ES/MS) on DC-CAS reading Total # of reading proficientin the 3rd grade Total # of 3rd graders tested *100 Gateway DC-CAS Proficient in 8 th grade math (ES/MS) % of 8 th grade students scoring at least Proficient on DC-CAS math in the most recent year Components 8 th grade DC-CAS scores Method Total # of math proficient in the 8th grade *100 Total # of 8th graders tested Critical gateway year for adolescent number skills Gateway Graduation rate (HS) Graduation rate of high school students based on OSSE standards Components Original 9 th grade cohort size for current 12 th grade # of graduates self-reported and validated through transcript reviews # of dropouts in the 12 th, 11 th, 10 th, and 9 th grades Method Total # of seniorsfor a SY with a regular diploma) *100 ( # of graduatesfor thesy + dropouts for latest 4 years period) Additional notes Drop outs are those students who leave schools for following reasons: (a) Whereabouts unknown (b) Work, (c) Voluntary, (d) Adult Education, (e)non-attendee, (f) No Show Aligned with OSSE standards Regular Diploma (OSSE) excluded the following: (a) GED, (b) certificates of completion, (c) certificates of attendance, (d) or, any other diploma not fully aligned with the State s academic records. Note: This definition may change starting w.aspx?agency=seo&section= 2&release=20924&year=2010 &file=file.aspx%2frelease%2f2 0924%2fDistrict_of_Columbia _Adjusted_Cohort_Graduatio n_rate_guidance_december_ 2010.pdf 30

31 Technical Guide Indicator Measure Description Components, method, additional notes Rationale Gateway 12 th grade SAT scores (HS) % of current 12 th grade students scoring at least 800 on the SAT Components SAT/ACT data for current 12 th grade SAT/ACT data for previous year 11 th grade Map of previous year 11 th grade to current 12 th grade 12 th grade enrollment in the most recent year Additional notes Only current 12 th graders are counted, but students scoring above 800 in earlier grades and not in 12 th grade will still contribute to score. In terms of ACT, students scoring above or equal to 16 are counted as on the numerator. Any HS with the 12 th grade that doesn t offer the SAT or ACT exams will be given 0 point on this metric. Method 11 th and 12 th grade students mapped together to determine overlap College Board research indicates that a score of at least 800 on the math and verbal sections of the SAT predicts with high likelihood a GPA of at least 2.0 in freshman year at community college or university For reference visit: ard.com/profdownload/pdf/r N-30.pdf SAT-ACT concordance table: ordance/ ( # of current 12th gradersscoringat least 800(V+ M)in SAT+ 16or hihger (V + M)in ACT) (# of 12th gradestudents) *100 Gateway 11 th grade PSAT scores (HS) % of 11 th grade students scoring at least 100 on the combined math and verbal portion of the PSAT Components PSAT data for current 11 th grade 11 th grade enrollment in most recent year Any HS with the 11 th grade that doesn t offer the PSAT exam will be given 0 points on this metric. Additional Notes 9 th and 10 th grade PSAT scores do not count because they are standardized to a different scale Method ( # of current 11th graders scoring at least 80 (V + M) on P SAT) (# of 11th grade students) * th grade PSAT performance of at least 80 on the math and verbal sections is a predictor of SAT performance of at least 800 on the math and verbal sections. For reference visit: ard.com/profdownload/psat- College-Readiness- Benchmark.pdf 31

32 Technical Guide Indicator Measure Description Components, method, additional notes Rationale Gateway College Acceptance (HS) % of high school seniors accepted to a full-time college or university Components Unique college acceptances self reported, verified by PCSB acceptance letter review 12 th grade enrollment in most recent year Method (#of seniorsacceptedto a full timecollegeor university) *100 (# of seniors) College acceptance is a necessary step to college enrollment and has been defined to be an important measure of student success Leading Attendance (ES/MS) Total Daily Attendance rate for 3 rd to 8 th grade. Components Attendance by grade-span Method (# of days present + # of excused absences) *100 (# days enrolled) Aligns with the State calculation. Only 3 rd through 8 th graders are included. Leading Attendance (HS) Re-enrollment (ES/MS & HS) Total Daily Attendance for 9 th to 12 th grade Re-enrollment rate for those eligible to return # of students ineligible to reenroll are: (a) Students in the exit grade (b) students who moved out from DC. (c) students in the non standard school(ec, Adult Components Attendance by grade-span Method (# of days present + (# days # of excused enrolled) absences) * 100 Components Previous fall enrollment by student id and grade (Audited Oct 5) Current fall enrollment by student id and grade (Audited Oct 5) Method Map students in previous fall and current fall to determine overlap Determine which students in previous falls are ineligible to reenroll (those in their final year; moved out from DC; enrolled in Early Childhood, Adult and GED frameworks) (Total # of Students attending both previous Fall and current fall) *100 (# of students in previous Fall) - (# of students ineligible to enroll) Aligns with the State calculation. Only 9 th through 12 th graders are included. Used to assess student and parent satisfaction and dropout rate Excluded are from early childhood and non-ged Adult Ed. because students are likely to leave for reasons other than dissatisfaction with school 32

33 Technical Guide Indicator Measure Description Components, method, additional notes Rationale and GED) Leading 9 th grade credits (HS) Components Review of all 9 th grade transcripts on-site review Percent of 9 th grade students receiving enough credits to be on track to meet OSSE graduation requirements in 4 years Method ( # of 9th track to grade students receiving enough credits to meet LEA graduation requirements (Total in 4 years) # of 9th grade students) be on ) *100 OSSE requirements: Students must pass 6 credits or the equivalent, consisting of Algebra I, an English course, a Science course, a Social Studies course that is among World History, US History, US Government or DC History, and two additional elective courses. LEA requirements vary by LEA. Some schools have adopted different internal standards for credit than OSSE 33