Report to Congress on Head Start Monitoring FISCAL YEAR 2009

Transcription

1 Report to Congress on Head Start Monitoring FISCAL YEAR 2009

2 TABLE OF CONTENTS EXECUTIVE SUMMARY... 1 I. Head Start Program Services... 4 II. Monitoring of Head Start Grantee Organizations BASIC MECHANICS OF THE MONITORING PROCESS EXCEPTION-BASED REPORTING MAJOR EFFORTS TO RESHAPE AND IMPROVE MONITORING SINCE FY REAUTHORIZATION OF THE HEAD START PROGRAM KEY CHANGES IN PROGRAM MONITORING EFFECTIVE IN FY III. Head Start Monitoring Reviews Conducted in FY IV. Compliance Status of Grantees That Underwent Reviews V. Overall Analysis of Findings for Reviewed Grantees VI. Analysis of Grantees with Noncompliances and Deficiencies VII. Deficiencies Requiring Immediate Corrective Action VIII. Analysis of Findings and Performance Standards Cited in Triennial and First-Year Reviews in FY IX. Corrective Actions and Their Outcomes X. New Directions in Monitoring for FY Appendix: Glossary Monitoring Report To Congress for FY 2009 TOC i

3 TABLE OF EXHIBITS Exhibit 1: Number of Reviews Conducted in FY 2009 by Review Type Exhibit 2: Grantees Reviewed in FY 2009: Program Type by Funded Enrollment Exhibit 3: Review Outcomes by Fiscal Year Exhibit 4: Compliance Status of Grantees Undergoing Triennial or First-Year Reviews Exhibit 5: Compliance Status by Grantee Type Exhibit 6: Compliance Status by Funded Enrollment Size Exhibit 7: Distribution of Grantees Reviewed by Number of Findings Exhibit 8: Average Number of Findings per Reviewed Grantee by Fiscal Year Exhibit 9: Distrbution of Head Start, Early Head Start, & HS/EHS Grantees by Number of Findings Exhibit 10: Distribution of Grantees with One or More Noncompliances by Number of Noncompliant Findings Exhibit 11: Distribution of Grantees With One or More Deficiencies by Number of Deficient and Noncompliant Findings Exhibit 12: Distribution of Grantees With One or More Deficiencies by Deficiency Type Exhibit 13: Deficiencies in FY 2009 Triennial and First-Year Reviews by Finding Category and Corrective Action Timeframe Exhibit 14: Distribution of Grantees with Noncompliant and Deficient Findings by Protocol Section Exhibit 15: Distribution of Findings by Protocol Section Exhibit 16: Performance Standards Most Frequently Cited as a Noncompliance Exhibit 17: Performance Standards Most Frequently Cited as a Deficiency Exhibit 18: Outcomes of Follow-Up Actions on Deficient Grantees that Received On-Site Follow-Up Reviews Exhibit 19: Outcomes of Follow-Up Actions on Grantees With One or More Deficiencies that Received Desk Reviews Exhibit 20: Outcomes of Follow-Up Actions on Grantees With One or More Noncompliances that Received On-Site Follow-Up Reviews Exhibit 21: Outcomes of Follow-Up Actions on Grantees With One or More Noncompliances that Received Desk Reviews Monitoring Report To Congress for FY 2009 TOC i

4 EXECUTIVE SUMMARY T he purpose of Head Start monitoring is to assess grantee compliance with requirements governing Head Start programs, including those specified in the Head Start Act, Head Start Program Performance Standards (45 CFR Parts and 45 CFR Parts 74 and 92) and other applicable Federal, State, and local regulations. Section 641A(f) of the Head Start Act requires a summary report be published at the end of each Federal fiscal year on the findings of monitoring reviews and outcomes of Quality Improvement Plans. This report describes such findings and outcomes for Fiscal Year (FY) Section 641A of the Head Start Act requires that each Head Start grantee receive a full on-site review immediately after completion of their first year of providing Head Start services and at least every three years thereafter. Follow-up reviews must be conducted for grantees that are found in the Triennial or First-Year review not to have met program requirements. Reviews are conducted by a team of reviewers knowledgeable about Head Start and led by a Review Team Leader (RTL), using the Office of Head Start (OHS) Monitoring Protocol, which employs a systems approach as a framework for assessing program services and quality. The Head Start Act also mandates that a grantee determined to have a non-compliance or a deficiency must correct these findings. If an area of non-compliance is not corrected in the specified period of time, it becomes a deficiency. Deficiencies must be corrected: 1) immediately, if the Secretary finds that the deficiency threatens the health or safety of staff or program participants or the integrity of Federal funds; or 2) within a period not to exceed one year, under a Quality Improvement Plan. If the grantee does not correct the deficiency within one year, OHS initiates the termination process or the grantee may relinquish the grant. If children or staff members are determined to be in imminent danger with no immediate solution, OHS may suspend the program, assign an interim provider so that services are not interrupted and only permit the program to reopen when the problem has been resolved satisfactorily. In Fiscal Year 2009, 986 unique Head Start grantees underwent Triennial (468) reviews; First- Year reviews (7); Other reviews (15), which typically are triggered when OHS becomes aware of a potential operational issue or concern and initiates a review that is outside the routine cycle; and Follow-up reviews (496). Review results were available for all 475 grantees that underwent Triennial or First-Year reviews. Approximately one-fifth of all grantees that underwent either a Triennial or a First-Year review in FY 2009 were found to be fully compliant with all Head Start requirements (96 of 475, 20.2%). Most grantees that underwent either a Triennial or a First-Year review in FY 2009 were found to have at least one area of noncompliance (ANC) cited in a Review Report (357 or 475, 75.2%), nearly half of these grantees had one or two areas of noncompliance. Approximately one of every 20 grantees reviewed was found to have one or more deficiencies in a Triennial or First- Year review in FY 2009 (22 of 475, 4.6%). Monitoring Report To Congress for FY 2009 Page 1

5 As was the case in FY 2008, the most common noncompliant findings in FY 2009 were associated with grantees failure to: assure that each staff member has an initial health examination; perform annual performance reviews of each Early Head Start and Head Start staff member; include number of children with disabilities in the community assessment; and, provide for the maintenance, repair, safety, and security of all facilities, materials, and equipment. The most common deficiencies involve a threat to children s health and safety, critical issues related to program governance, a substantial failure to perform, misuse of Head Start funds, or loss of legal status. OHS revises the Head Start monitoring process each fiscal year to foster evidence-based, quality programming; improve efforts to measure quality; address program integrity; and to reflect the higher standards expected of programs. For example, in FY 2009, OHS piloted the Classroom Assessment Scoring System (CLASS) instrument as a method of monitoring teacher effectiveness and classroom quality. In addition, OHS has increased oversight and reviews of programs with identified risk factors; and notified grantees that it will conduct unannounced monitoring visits. In FY 2009, OHS continued its enhancement, expansion, and centralization of the monitoring system by refining the Monitoring Protocol; planning and implementing several procedural and systems changes and improvements that govern collection of data in the field and the issuance of Review Reports; and testing other innovations of process with respect to deploying review teams. Additionally, OHS assumed a central role in orchestrating and managing a national schedule of Follow-up reviews (OHS centralized the scheduling of Triennial and First-Year Reviews in 2006.) OHS also broadened its use of data in FY 2009 to improve the integrity, timeliness, and quality of information regarding grantee performance, as well as to track performance over time. During FY 2009 and FY 2010, HHS was developing proposed regulations to implement two provisions of the 2007 reauthorization of the Head Start Act: 1) the requirement that established Head Start grantees will be awarded grants for a five-year period and only grantees determined to be delivering high-quality services will be given another five-year grant noncompetitively; and 2) the requirement to develop a designation renewal system to determine if a Head Start agency is delivering a high-quality and comprehensive Head Start program that meets the educational, health, nutritional, and social needs of the children and families it serves, and meets program and financial management requirements and standards. HHS issued proposed regulations that articulate the details of the proposed designation renewal system (DRS) in September On November 9, 2011 the final DRS was published in the Federal Register and it became effective on December 9, HHS plans to modify its monitoring system as appropriate based on the implementation of the DRS. During FY 2010, while preparing to implement other improvements to monitoring, OHS contemplated specific changes in policy and procedure for FY 2010 and beyond that would ensure compliance with certain provisions of the Head Start Act that were established in the 2007 reauthorization of the Head Start Act, as well as Family Child Care Regulations (published in February 2008), and the American Recovery and Reinvestment Act (enacted in February 2009), which enabled program expansion. Changes to monitoring implemented in FY 2010 were made in five areas, including: Monitoring Protocol and software; system changes resulting from Monitoring Report To Congress for FY 2009 Page 2

6 a Government Accountability Office (GAO) investigation; full-scale implementation of the Classroom Assessment Scoring System (CLASS) instrument; program expansion enabled by the American Recovery and Reinvestment Act (ARRA); and new or expanded legislative or regulatory requirements. OHS also expanded the use of unannounced reviews in FY 2011 to include approximately 10% of all grantees eligible for a Triennial or First-Year review. Monitoring Report To Congress for FY 2009 Page 3

7 I. Head Start Program Services Head Start, created and first authorized in 1965 under the Head Start Act (42 USC 9801, et seq.), is a national program that provides comprehensive child development services primarily to lowincome children (ages three to five) and their families, with a special focus on helping children develop the early literacy and numeracy skills they need to succeed in school. In 1994, Head Start was authorized to serve children birth to three in response to mounting evidence that the earliest years matter a great deal to a child s growth and development. Early Head Start provides services to infants and toddlers, from birth to age three, as well as to pregnant women. Head Start promotes school readiness by enhancing the social and cognitive development of children through educational, health, nutritional, social, and other services. It also recognizes the important role of parents and encourages them to participate in a variety of activities and experiences that support and foster their children s development and learning, and helps them progress toward their educational, literacy, and employment goals. Head Start also requires programs to provide opportunities for parental involvement in the development, conduct, and governance of local programs through participation in policy groups (e.g., Policy Councils). Head Start is administered by the Office of Head Start (OHS) of the Administration for Children and Families (ACF) within the U.S. Department of Health and Human Services (HHS). Grants are awarded through the ACF Regional Offices and the Office of Head Start s American Indian-Alaska Native and Migrant and Seasonal Programs Branches directly to local public agencies, private organizations, Indian tribes, and school systems for the purpose of operating Head Start programs at the community level. Monitoring Report To Congress for FY 2009 Page 4

8 II. Monitoring of Head Start Grantee Organizations The purpose of Head Start monitoring is to assess grantee performance and compliance with requirements governing Head Start programs, including those specified in the Head Start Act, Head Start Program Performance Standards, and other applicable Federal, State, and local regulations and to facilitate, if warranted, program improvements. The Head Start Act mandates that each Head Start grantee receive a monitoring review at least once every three years, that each newly designated grantee be reviewed after the completion of its first year (and then at least every three years thereafter), and that Follow-up reviews be conducted for all grantees that fail to meet the standards. In FY 2009, reviews were conducted by a team of reviewers knowledgeable about Head Start and led by a Review Team Leader (RTL). Each review was guided by a standard Protocol, which reviewers used universally to assess program performance and compliance. The following sections describe the basic mechanics of the monitoring process; a description of the exception-based reporting system OHS uses; significant steps OHS has taken over the last few years to improve how the process works; the effect of the 2007 reauthorization of the Head Start Act on monitoring; and key changes to monitoring that were made effective in FY Basic Mechanics of the Monitoring Process The monitoring process used a comprehensive approach to confirm that grantees complied with Federal legislative, regulatory, and program requirements. Triennial or First-Year on-site monitoring reviews were conducted by a team of qualified non-federal consultants supervised by a RTL. Prior to the start of the fiscal year, OHS sent a global letter to all grantees scheduled for a First-Year or Triennial review. Then, 30 days prior to the on-site review, grantees were sent written notification of the specific date of the review. Soon after this official written notification of the review date was received, the RTL contacted the grantee to begin scheduling on-site activities. Prior to the on-site review, team members reviewed grantee documents posted online. Generally, Head Start monitoring reviews were scheduled for a four- to five-day period and were conducted by teams of seven to eight reviewers. While every review was conducted within the framework of the Protocol, larger grantees, including those with delegate agencies and with complex program designs (e.g., grantees with both Head Start and Early Head Start programs) sometimes required larger review teams. A category of the very largest grantees, considered super grantees, required both substantially larger review teams and longer review periods. Smaller grantees allowed for smaller teams of reviewers. Once on site, the review team initiated the information collection process, which was supported by the Protocol, checklists, and software. Review teams relied on multiple modes of inquiry interviews, observations, documentation review, and analysis to consider grantee compliance with program requirements. The RTL facilitated nightly team meetings to discuss and document preliminary findings, and to identify areas requiring further exploration. The on-site review culminated in the development of a preliminary report of findings. At the conclusion of the onsite review, the RTL reviewed the totality of evidence, made preliminary decisions of noncompliance, and submitted the preliminary draft report to OHS. Monitoring Report To Congress for FY 2009 Page 5

9 2. Exception-Based Reporting OHS utilizes a system of exception-based reporting to comply with the Federal mandate to inform grantees of findings that should be corrected (Section 641A(e) of the Head Start Act, as amended in 2007). Fundamental to the exception-based reporting process was the collection, verification, and substantiation of evidence from multiple sources to support findings of noncompliance. Review teams conducted interviews with program staff, policy council and board members, and others; observed children and teachers in their natural settings; and reviewed program documents and materials, as well as children s files to assess compliance with Head Start requirements. Team members were encouraged to share information on a routine basis through the software application, team meetings, electronic mail, and telephone communications throughout the day. Based on the analysis of the evidence and the team s recommendations, the RTL rendered preliminary decisions regarding grantee compliance with program requirements. An initial finding identified by the review team was referred to as a preliminary area of noncompliance (PANC). To support each preliminary area of noncompliance, the review team was required to cite at least one Head Start requirement and provide sufficient, well documented evidence and descriptions of the problem cited. If during an on-site review the RTL identified a deficiency that required immediate corrective action, an HHS responsible official provided written notice of deficiency requiring immediate correction and the RTL was authorized to direct the grantee to take immediate corrective action to ensure that staff and/or children were removed from imminent harm or immediate danger and that the cause of the imminent harm or immediate danger was corrected. The corrective action required of the grantee to correct the immediate deficiency was provided in the notice. Deficiencies. The Head Start Act, as amended in 2007, defines a deficiency (Section 637 [42 USC 9832]) as follows: (A) Systemic or substantial material failure of an agency in an area of performance that the Secretary determines involves: (i) A threat to the health, safety, or civil rights of children or staff; (ii) A denial to parents of the exercise of their full roles and responsibilities related to program operations; (iii) A failure to comply with standards related to early childhood development and health services, family and community partnerships, or program design and management; (iv) The misuse of funds received under this subchapter; (v) Loss of legal status (as determined by the Secretary) or financial viability, loss of permits, debarment from receiving Federal grants or contracts, or the improper use of Federal funds; or (vi) Failure to meet any other Federal or State requirement that the agency has shown an unwillingness or inability to correct, after notice from the Secretary, within the period specified; Monitoring Report To Congress for FY 2009 Page 6

10 (B) Systemic or material failure of the governing body of any agency to fully exercise its legal and fiduciary responsibilities; or (C) An unresolved area of noncompliance. Examples of common deficiencies constituting a systemic or substantial material failure of an agency in an area of performance, also known as a failure to perform, include failure to establish and implement procedures for ongoing program monitoring or failure to conduct a criminal background check. In the health and safety area, an example is the failure to develop and implement within 90 days of a child s entry into the Head Start program a follow-up plan to ensure that any necessary medical treatment has begun. Some deficiencies might require immediate corrective action, such as improper storage or preparation of food or failure to ensure proper supervision of children at all times. Areas of Noncompliance. The Performance Standards, at 45 CFR (a), authorize OHS to determine, on the basis of the review, if grantees have areas of noncompliance that do not constitute deficiencies, but must, nonetheless, be corrected. Examples of common noncompliances are associated with failure on the part of a grantee to: assure that each staff member has an initial health examination; perform annual performance reviews of each Early Head Start and Head Start staff member; include in the community assessment an estimated number of children with disabilities; and, provide for the maintenance, repair, safety, and security of all facilities, materials, and equipment. Findings from a review, as required in the Act, are to be presented to the Head Start agency in a timely, transparent, and uniform manner that can assist with program improvement and be used by the agency to inform development and implementation of an appropriate plan for training and technical assistance. 3. Major Efforts to Reshape and Improve Monitoring Since FY 2006 Over the last several years, OHS has improved its monitoring system significantly to ensure national consistency and objectivity in grantee assessments across regions, provide grantees with monitoring findings in a transparent and uniform manner, and provide a national system for collecting, maintaining, and analyzing data related to the monitoring process. Key recent improvements, initiated over the period from FY 2006 through FY 2008, are described briefly below. This section is followed by a description of key changes to monitoring procedures or systems. These were initiated and made effective in FY Established National Schedule of Monitoring Reviews. In FY 2006, OHS established and assumed oversight of a new centralized process for scheduling and planning all Triennial and First-Year reviews. A national pool of RTLs was created to supervise Triennial and First-Year reviews. OHS established a general rule that this national reviewer pool would be scheduled to lead review teams outside of their home region in an effort to minimize any perception of subjectivity and to increase national consistency in Head Start monitoring. Additionally, in order to increase comprehensiveness in the review process, OHS directed that for Triennial or First- Year reviews, grantees with delegate agencies were required to have data collected from every delegate (e.g., staff positions, service plans, self assessment). Professionalized the Reviewer Pool. Over the last several years, OHS has devoted Monitoring Report To Congress for FY 2009 Page 7

11 considerable effort to ensuring that each review was staffed by individuals knowledgeable about Head Start programs. This effort included, in particular, establishing specific content area qualifications, and assuring minimum standards for experience and educational requirements of reviewers. With the objective of continuing to strengthen the integrity of the reviewer pool across the monitoring process, OHS implemented a number of policies and procedures to guide assignment of individual reviewers, pre-review preparation of reviewers, and post-review learning and improvement. A new governing framework was established, which limited the number of reviews that reviewers employed by a Head Start grantee or delegate agency could participate in each year and prevented reviewers from reviewing programs within their home States. To further prepare reviewers, OHS established a pre-site process for providing review team members with a standard set of grantee documents for review in advance of the site visit, available via the Web. OHS also established weekly pre- and post-review team briefings. Through post-review briefings, OHS could identify what processes needed to be strengthened or where additional supports were required to facilitate the reviewers work while on site. Implemented Monitoring Software. To facilitate the collection, aggregation, analysis, and reporting of monitoring data and to improve the detail, specificity, and clarity of Review Reports, OHS developed and launched application software in FY 2006 to manage more effectively the monitoring system and produce reports on the nation s Head Start agencies and programs. The OHS Monitoring Software (OHSMS): Standardized report writing processes and the Head Start Review Report; Ensured that Head Start Review Reports received by grantees document the findings of the on-site review in a clear, consistent format organized to facilitate decision and action; Facilitated the review process by providing ready access to: all program monitoring instruments and tools, including service area protocols, core questions, the full set of standards, and other monitoring tools; report information, and information regarding corrective actions, strengthening OHS management of follow-up review activity; Provided a centralized repository of review information for enhanced data aggregation, analysis, and reporting; Increased automation of the monitoring process, as well as standardization of the Head Start Review Report; Established a foundation for a performance-based management system that moved OHS forward on the principles of quality assurance, accountability, and continuous quality improvement. Centralized Quality Control and Finalization of Review Reports. OHS moved toward ensuring national consistency across monitoring reviews and reports by centralizing the quality control and compliance determination processes. This change shifted responsibility for the form, content, and issuance of monitoring reports from individual Regional Offices to OHS Monitoring Report To Congress for FY 2009 Page 8

12 central office. OHS assumed responsibility for the quality assurance process to ensure that Head Start Review Reports submitted by review teams following the on-site review met rigorous standards for accuracy, clarity, and legal soundness. Centralization of quality control and the heavy emphasis on evidence-based findings provided increased consistency in the quality, detail, specificity, and utility of Head Start Review Reports, as well as increased timeliness in issuing monitoring Review Reports to grantees, thereby enabling grantees to take corrective action and bring their program into compliance more quickly. Implemented a Desk Review as a Method of Follow-Up. Effective in FY 2007, a Program Specialist, operating from a Regional Office and having oversight responsibilities for a caseload of grantees, may request a Desk review be conducted as a method of follow-up for any grantee with areas of noncompliance where correction of such findings can be verified by a review of that grantee s appropriate program documents. A Desk review of program documents enables the Program Specialist, rather than an on-site review team, to verify that grantees have taken appropriate corrective action. The Program Specialist initiates a Desk review in the OHS Monitoring Software for a particular grantee by selecting which areas of noncompliance require confirmation as having been corrected. Prior to initiating the Desk review, the Program Specialist collects sufficient evidence (e.g., documents, pictures) to support the request. Each request for a Desk review must be approved by the Regional Program Manager, who reviews the evidence for sufficiency and can approve, deny, or request additional information of the Program Specialist. When appropriate, a Desk review method of verifying corrective action, in place of an on-site follow-up team, enables OHS to increase efficiencies and decrease costs. Continued To Emphasize the Importance of Pre-site Review Planning. OHS continued to emphasize the pre-site visit planning stage as a critically important component of an effective review. In addition to changes implemented in FY 2007 and FY 2008, including increased presite conversations with the grantee and the pre-site availability of grantee documents, OHS required that all team members read these documents to ensure their familiarity with the grantee and improve the efficiency and effectiveness of the on-site phase of the review. 4. Reauthorization of the Head Start Program On December 12, 2007, several months into the FY 2008 monitoring season, the Improving Head Start for School Readiness Act was signed into law, reauthorizing the Head Start program through September 30, Two provisions and a modified definition of deficiency established by the new law were incorporated into program operations and the monitoring process. First, OHS immediately moved to include the new eligibility requirements, including authorizing Head Start and Early Head Start grantees to serve up to 35 percent of their enrollment with children from families with incomes up to 130 percent of the Federal poverty line. When a grantee was able to demonstrate sufficient cause, OHS also began permitting a grantee to reduce its funded enrollment, after submitting a proposal to its Regional Office for review and approval. 1 The Head Start Act of 1998 (Public Law ) was amended by the Improving Head Start for School Readiness Act of 2007 (Public Law ). Monitoring Report To Congress for FY 2009 Page 9

13 Many provisions in the new law, however, required additional time for contemplation, determinations as to whether or not additional guidance was necessary, and issuance of regulations prior to incorporation into the monitoring system. OHS encouraged all Head Start programs to read and become familiar with requirements in the new law and to review regularly the Policy Clarification website at Head Start s Early Childhood Learning and Knowledge Center (ECLKC), as it unveiled guidance on the new requirements throughout FY 2008 and FY Additional provisions will be incorporated into the monitoring process in the future as appropriate. 5. Key Changes in Program Monitoring Effective in FY 2009 OHS continued its enhancement, expansion, and centralization of the monitoring system in FY 2009 by refining the Monitoring Protocol; planning and implementing several procedural and systems changes and improvements that govern collection of data in the field and the issuance of Review Reports; and testing other innovations of process with respect to deploying review teams. Beginning in FY 2009, OHS assumed a central role in orchestrating and managing a national schedule of Follow-up reviews (OHS centralized the scheduling of Triennial and First- Year reviews in FY 2006). OHS also broadened its use of data in FY 2009 to improve the integrity, timeliness, and quality of information regarding grantee performance, as well as to track performance over time. The following sections describe the major changes implemented by OHS for the FY 2009 program year, as well as other key activities related to monitoring initiated in FY Refined the Monitoring Protocol. The Monitoring Protocol, designed to guide a more focused, efficient and comprehensive assessment of grantee compliance, significantly contributed to the enhanced consistency and accountability of the overall OHS monitoring system. First released for the FY 2007 monitoring season, this integrated Protocol is comprised of compliance questions that cover all program service areas and management systems. Each compliance question is linked directly to a Head Start performance standard; therefore, reviewers evaluate any grantee information collected during the review activities against Federal regulations and Head Start Act requirements. During FY 2008, Review Team Leaders and reviewers provided OHS with continuous feedback on the Protocol s effectiveness in monitoring grantee compliance. Head Start grantees also provided valuable feedback on the quality and conduct of the more than 800 reviews throughout the year. In preparation for the FY 2009 monitoring season, targeted focus groups with Review Team Leaders, reviewers, and content area experts were held to identify strengths and weaknesses in the Protocol s content and process. Data from these focus groups were used to refine further the Protocol content and structure. The most significant change to the Protocol and its supporting software in FY 2009 was the addition of targeted questions and guides to provide the reviewers with specific guidance regarding what to ask grantees, what to look for when reviewing documents, and what to observe during on-site visits to ensure thorough collection of the information needed for OHS to make a determination regarding each compliance question. Targeted questions, which clearly define what information reviewers must collect, are organized into guides for each review Monitoring Report To Congress for FY 2009 Page 10

14 activity (interviews, observation, document review, etc.). While completing a review activity, reviewers enter responses to each targeted question in the guide for that review activity. These notes automatically appear beneath any relevant compliance questions in the software, allowing reviewers to use these notes as evidence to answer compliance questions. By requiring review teams to adhere to a uniform and defined set of compliance and targeted questions, the FY 2009 Protocol more clearly defines review activities and standardizes the set of information collected during each review, thereby increasing the focus, efficiency, fairness, and comprehensiveness of the review. In FY 2009, OHS began requiring review teams to identify any program strengths for each grantee reviewed. Review teams identified strengths in two ways: providing Head Start directors the opportunity to identify their own program strengths and/or encouraging reviewers to identify program strengths through observations or other review activities. For each strength identified, reviewers collected supporting evidence and wrote strength narratives that were incorporated into the final monitoring Review Report. In addition to strengths, the OHSMS was modified to enable reviewers to indicate concerns. When reviewers found an issue related to a specific targeted question, reviewers marked this in the software as a concern, which allowed the reviewer and the review team to track grantee issues and identify patterns. When a reviewer indicated a concern, but ultimately determined the grantee was in compliance with the associated performance standard, reviewers were required to indicate why the concern did not warrant a noncompliance. Implemented New Procedure for Identifying and Creating an Immediate Deficiency. During an on-site review, a review team may observe conditions that are severe enough to constitute an immediate deficiency. In such circumstances, OHS is obliged to issue to the grantee a Notice of Immediate Deficiency, requiring either immediate corrective action, correction while the review team remains on site, or correction within an identified timeframe following receipt of the Notice (e.g., 10 days or 30 days). Reviewers are encouraged to raise with the Review Team Leader (RTL) any concerns that might trigger a Notice of Immediate Deficiency, and, in the process, be able to document fully the issue as well as articulate the risk it poses or the real or potential impact. Examples of immediate deficiencies include: Fiscal issues, such as evidence of fraud, misuse, or falsification of fiscal records; Health issues, such as evidence of mold, unsanitary conditions, contamination of food, or animal waste, or significant medical issues where follow-up care has not been provided; Safety issues, such as lack of criminal background check for staff, imminent risk of injury or death to children or staff, or observation of maltreatment or abuse of children. In FY 2009, OHS implemented a procedure supported by the software to identify and create the immediate deficiency, and, once reported to OHS, facilitate subsequent steps. Upon discussion with the RTL, reviewers document evidence to support the concern and draft a preliminary area of noncompliance (PANC) in the OHSMS. Via the OHS Help Line, the RTL informs OHS that a potential immediate deficiency has been identified. OHS reviews the PANC to determine Monitoring Report To Congress for FY 2009 Page 11

15 whether it meets the criteria for an immediate deficiency. If substantiated by OHS, the Notice is generated automatically, signed, and delivered to the grantee. Next, the finding is subject to follow up by the review team in accordance with the terms under which it was issued. Included Findings Corrected On Site in the Review Report. Once a review team identifies a concern, the grantee is encouraged to take corrective action as soon as possible. In some instances, a grantee may be able to address and correct a concern while the review team remains on site. These kinds of concerns are referred to as findings corrected on site (COS). While findings corrected on site if substantiated as actual findings following the review by OHS had been excluded from the Head Start Review Report in FY 2007 and FY 2008, OHS began again to include such findings in the Review Report effective in FY Focused on Reducing the Number of Dropped Findings. Periodically, evidence collected by review teams is not sufficient to substantiate a finding. During the quality assurance process, if additional evidence cannot be obtained, the finding must be dropped. In FY 2009, OHS concentrated on reducing the number of findings identified during reviews that later are dropped by addressing several areas of the process that need improvement. Through training, increased communication with reviewers, and enhanced feedback, OHS focused specifically on: 1) reducing instances of contradictory evidence, or insufficient documentation of evidence; 2) reducing mismatches between evidence provided and citations; and 3) increasing the quality of feedback Review Team Leaders provide review team members regarding the materiality of the evidence collected. Piloted the Classroom Assessment Scoring System (CLASS) Instrument. Also in FY 2009, to meet requirements added in the 2007 reauthorization of the Head Start Act, OHS pilot-tested the Classroom Assessment Scoring System (CLASS) instrument as a method of monitoring teacher effectiveness and classroom quality. Developed by the University of Virginia and validated in over 2,000 preschool classrooms, CLASS is an observational instrument that assesses interactions between children and teachers in three broad domains, including emotional support, classroom organization, and instructional support. In preparation for implementation in FY 2010, OHS conducted regionally-based trainings on CLASS for education managers and supervisors in Head Start programs serving preschool children ages three to five. The purpose of the training was to demonstrate how CLASS could be used to assess and improve preschool classroom quality, as well as to link the instrument with professional development. OHS also began certifying Education and Early Childhood Development Services (ECD) reviewers to implement CLASS during monitoring reviews through two-day CLASS sessions throughout the country. These sessions were designed to enable reviewers to practice using the instrument by observing and assessing videos of preschool classroom environments. Modified Procedures Governing Follow-up Reviews. Also effective in FY 2009, OHS established that the due date for a grantee s Follow-up review was the latest possible correction action deadline, and Follow-up teams were to review all findings during that inclusive review. 2 2 Grantees can have findings with different corrective action timeframes. In such cases, the due date for the grantee s Follow-up review will correspond with the later timeframe. Monitoring Report To Congress for FY 2009 Page 12

16 To build on experiences initiated in FY 2008, OHS also expanded the use of Desk reviews in FY 2009 to review findings for which correction can be verified remotely through interviews and documents. III. Head Start Monitoring Reviews Conducted in FY 2009 This section presents basic descriptive data on Head Start monitoring reviews conducted in FY 2009, specifically addressing the following questions: How many and what types of monitoring reviews were conducted in FY 2009? Which reviews are highlighted in this Report to Congress? How Many and What Types of Monitoring Reviews Were Conducted In FY 2009? OHS conducted a total of 986 monitoring reviews from October 1, 2008 through September 30, A conducted review is defined as a review for which OHS had selected and deployed a team to review a grantee within the fiscal year. Reviews conducted by OHS consist of four primary types: First-Year, Triennial, Other, and Follow-up. First-Year reviews are reviews conducted on grantees after one year of operation, while Triennial reviews are conducted once every three years throughout a grantee s life cycle. First-Year and Triennial reviews are therefore routine monitoring reviews conducted at planned and scheduled periods. Other reviews are non-routine in nature, and typically are triggered by a potential performance issue or concern in response to which OHS conducted an out-of-cycle review. Grantees found to have noncompliances or deficiencies in Triennial, First-Year, or Other Reviews underwent Follow-up reviews. Beginning in FY 2007, OHS also began utilizing a Desk review, which enabled OHS to follow up on certain grantees that had findings in their Triennial or First-Year review without having to deploy an on-site review team. The 986 total reviews conducted in FY 2009 included the following (Exhibit 1): 475 First Year and Triennial reviews, including: 15 Other reviews 7 First-Year reviews 468 Triennial reviews 496 Follow-up reviews, including: 317 Follow-up reviews conducted by an on-site review team 179 Desk reviews conducted without an on-site review team Of the 317 on-site Follow-up reviews conducted in FY 2009, 71 reviews were conducted on grantees whose Triennial or First-Year review also occurred during FY 2009 (i.e., October 1, 2008 September 30, 2009), while 246 reviews were conducted on grantees whose Triennial or First-Year review took place prior to FY 2009 (i.e., prior to October 1, 2008). Likewise, of the 179 Desk reviews conducted in FY 2009, 63 reviews were conducted on grantees whose Triennial or First-Year review also occurred during FY 2009, while 116 reviews were conducted on grantees whose Triennial or First-Year review took place prior to FY Monitoring Report To Congress for FY 2009 Page 13

17 The total number of reviews conducted in FY 2009 (n=986) is similar to the total conducted in FY 2008 (n=974); a total of 1,107 reviews were conducted in FY 2007, while 804 reviews were conducted in FY While fewer Triennial reviews were conducted in FY 2009 than in FY 2008 (468, compared with 552 in FY 2008), significantly more Follow-up reviews, including Desk reviews, were conducted in FY 2009 (496, compared with 398 in FY 2008). OHS expanded the use of Desk reviews in FY 2009, increasing in number from 90 in FY 2008 to 179 in FY Which Reviews Are Highlighted In This Report To Congress? This annual Head Start Monitoring Report to Congress for FY 2009 focuses on the cohort of grantees that underwent Triennial and First-Year reviews in FY 2009 and to which Review Reports were issued by March 1, 2010, as well as all Follow-up reviews on these FY 2009 reviews issued by March 1, 2010, including Desk reviews. 3 Outcomes of Follow-up reviews conducted on grantees with Triennial and First-Year reviews in FY 2009 are presented later in this report. This report does not track and report follow-up activity, including Desk reviews, on grantees whose initial Triennial and First-Year review preceded FY Grantees that underwent Other reviews in FY 2009 are not included in this analysis. Monitoring Report To Congress for FY 2009 Page 14

18 IV. Compliance Status of Grantees That Underwent First Year or Triennial Reviews At the end of the monitoring process, a Head Start Review Report was issued to each grantee to indicate the grantee s status in terms of its compliance with Head Start program requirements. The grantee s status is a function of the final determinations made by OHS on each of the preliminary findings documented by the review team during the on-site review. Each finding documented by a review team during a review, and subsequently validated by OHS, may be one of two types: noncompliant or deficient. If the Head Start Review Report issued to the grantee contained one or more noncompliant findings, but no deficient findings, the grantee was given a status of having one or more noncompliances. If the Head Start Review Report contained one or more deficient findings, the grantee was given a status of having one or more deficiencies ; grantees with this status may have also had one or more noncompliant findings. Preliminary areas of noncompliance that were identified by the review team while on site, but corrected by the grantee while the review was in progress, were not included in the Head Start Review Report, although they were maintained electronically in the grantee s history. To ensure that findings were addressed and corrected, Follow-up reviews were scheduled and conducted for any grantee found to have noncompliances or deficiencies; under certain conditions, OHS enabled grantees to affirm correction of findings via a Desk review. This section presents an analysis of the compliance status of grantees that underwent monitoring reviews in FY The questions addressed in this section are: Which types of grantees were reviewed in FY 2009? What was the overall compliance status of grantees following reviews in FY 2009? How did compliance status for grantees that underwent Triennial reviews compare with grantees that underwent First-Year reviews in FY 2009? How did compliance status compare by grantee type in terms of whether they provided Head Start services only, Early Head Start services only, or both services? How did compliance status of grantees compare for grantees by funded enrollment? Monitoring Report To Congress for FY 2009 Page 15

19 Which Types Of Grantees Were Reviewed In FY 2009? OHS conducted 468 Triennial and seven First-Year reviews on 475 unique grantees of nearly 1,600 total grant-holding organizations nationwide in FY 2009; review results are available in this report for all 475 grantees. Exhibit 2 presents the distribution of these 475 grantees by program type (Head Start, Early Head Start, both Head Start/Early Head Start), as well as the distribution of grantees reviewed by their funded enrollment sizes within each program type. More than one-half of grantees reviewed in FY 2009 (58.7%) provided only Head Start services to three to five year old children, while slightly less than 10 percent of grantees served only the zero-to-three population (7.4%); approximately one-third of grantees (33.9%) operated programs that served both age groups. Funded enrollment levels varied by program type. Grantees reviewed in FY 2009 that provided only Early Head Start services tended to be smaller, with a higher proportion of programs with funded enrollments of fewer than 100 children and no programs serving more than 300 children. Exhibit 2 also shows that programs providing both Head Start and Early Head Start services tended to serve greater numbers of children than those providing only Head Start services, as would be expected given the broader spectrum of services provided. By comparison, in FY 2008, grantees providing only Head Start Services were a somewhat smaller percentage of all grantees reviewed (54.2%), while grantees providing only Early Head services were a somewhat larger percentage of all grantees reviewed (9.6%). Monitoring Report To Congress for FY 2009 Page 16

20 What Was The Overall Compliance Status Of Grantees Following A Review In FY 2009? A grantee s status is determined by the type(s) of findings identified during a review, if any. As stated previously, a grantee determined to have one or more noncompliant findings but no deficiencies is considered to be a grantee with one or more noncompliances, while a grantee with one or more deficiencies has at least one deficient finding and may have one or more noncompliant findings. Approximately one-fifth of all grantees that underwent either a Triennial or a First-Year review in FY 2009 were found to be compliant with Head Start requirements (96 of 475, 20.2%). Most grantees that underwent either a Triennial or a First-Year review in FY 2009 were found to have at least one area of noncompliance (ANC) cited in a Review Report (357 of 475, 75.2%), although nearly half of these grantees had only one or two areas of noncompliance (numbers of findings are described later in this report). Approximately one of every 20 grantees reviewed was found to be have one or more deficiencies in a Triennial or First-Year review in FY 2009 (22 of 475, 4.6%). As indicated in Exhibit 3, the overall compliance status of grantees following a review was relatively constant from FY 2007 through FY 2009, with grantees slightly less likely to be compliant in FY 2009 than in FY 2008 and FY 2007 (20.2% were compliant in FY 2009, compared with 22.3% in FY 2008 and 22.0% in FY 2007). The most common review outcome for grantees reviewed in FY 2009, as in FY 2006, FY 2007, and FY 2008, was having one or more areas of noncompliance. The percentage of grantees found to have at least one area of noncompliance each year has increased since FY 2006 (from 58.9% in FY 2006 to 75.2% in FY 2009), while the percentage of grantees found to be have one or more deficiencies each year has declined significantly (from 32.8% in FY 2006 to 4.6% in FY 2009). Changes in review outcomes occurred, in part, due to a statutorily prescribed change in the definition of a deficiency. Section 637 of the Head Start Act, as amended December 12, 2007, tightened the definition of a deficiency to three specific types of violations. The full definition of a deficiency, as defined in the Act, can be found in the Glossary. Exhibit 3 Review Outcomes by Fiscal Year Review Outcome Fiscal Year FY 2006 FY 2007 FY 2008 FY 2009 N % N % N % N % Compliant % % % % One or more noncompliances One or more deficiencies % % % % % % % % Total % % % % Note: Figures for each fiscal year include outcomes for Triennial and First-Year reviews only. Figures for FY 2007 include two reviews that were outstanding upon publication of the Monitoring Report to Congress for FY Data on the outcomes of Follow-up reviews conducted on grantees that underwent Triennial or Monitoring Report To Congress for FY 2009 Page 17