325T Special Education Preservice Program Improvement Grants Webinar on Grant Performance Report for Continuation Funding

Transcription

1 325T Special Education Preservice Program Improvement Grants Webinar on Grant Performance Report for Continuation Funding Personnel Development Program Office of Special Education Programs US Department of Education Washington, DC February 24,

2 Today s Agenda Welcome from OSEP Today s Topic Grant Performance Report for Continuation Funding Overview of Performance Reporting Developing Performance Measures Completing Section A of the ED 524B Completing Sections B&C of the ED 524B Q & A after each part and further Q&A and Discussion Closing & Survey 2

3 GRANT PERFORMANCE REPORT FOR CONTINUATION FUNDING FY 2012 Office of Special Education Programs US Department of Education Personnel Development Program 325T 3

4 Annual Grant Performance Report (APR) An annual report of your activities and performance in meeting the approved objectives of the project and responsible use of federal funds Required for all active grants, including those in no cost extension (NCE) OSEP reviews the report to determine if substantial progress has been made in order to receive continued funding or a NCE 4

5 Overview 5 Recognize strong project objectives that can be associated with high quality performance measures Develop relevant, measurable, outcome oriented performance measures related to your objectives that maximize the potential for meaningful data reporting and positive outcomes Complete the ED Grant Performance Report (aka. APR) using form ED 524B.

6 Why Is This Important? 6 High quality objectives and measures Make it easier for you to measure your progress for the purpose of grant management Allow you to report progress easily and quantitatively Establish targets (both short-term/annual & longterm) Allow OSEP staff to gather evidence of program effectiveness Taken from the Center for Evaluation and Education Policy (CEEP) presentation at

7 Goal Objectives - Measures Program Goal Project Objectives What your project is doing to support the overall program goal (Found in your application) Performance Measures How you measure your progress toward meeting your objectives (Program/GPRA, Project) 7 Taken from the Center for Evaluation and Education Policy (CEEP) presentation at

8 Project Objectives What are you trying to accomplish? Objectives should answer this question. Preferred format for objectives: Begin the objective with a verb and define a desired outcome or condition 8

9 High Quality Project Objectives Relevance How relevant is the project objective to the overall goal of the program and/or the goal of your project? Applicability How applicable is the project objective to the specific activities that are being conducted through your particular project? 9 Taken from the Center for Evaluation and Education Policy (CEEP) presentation at

10 High Quality Project Objectives Focus How focused is the project objective? Measurability Are there concepts in the project objective that lend themselves to measurement? If so, is measurement feasible? 10 Taken from the Center for Evaluation and Education Policy (CEEP) presentation at

11 Project Objectives - Examples Implement a high-quality personnel development program that is aligned to State and national professional organization standards Prepare highly qualified personnel to serve children with high incidence disabilities and their families Provide ongoing support to retain scholars in the program Develop and offer courses in assistive technology that are evidence-based 11

12 Performance Measures How are you measuring your progress in meeting your objectives? Performance measures should answer this question. 12

13 Performance Measures Measurable indicator used to determine how well objectives are being met. How will progress be assessed? How much progress will constitute success? How will it be known if an objective or part of an objective has been achieved? 13 Taken from the Center for Evaluation and Education Policy (CEEP) presentation at

14 Performance Measures Performance Measure 1a Project Objective 1 Performance Measure 1b Performance Measure 1c 14 Taken from the Center for Evaluation and Education Policy (CEEP) presentation at

15 2 Types of Performance Measures Program All grantees funded under the Personnel Development Program (PDP) must report on the PDP program performance measures established by OSEP. Project Each grantee reports on the approved project performance measures established to meet their project objectives. 15

16 Program Performance Measures Program Measures were established by OSEP for the Personnel Development Program. Measures apply to all grants funded under the Personnel Development Program. Results on these measures are reported to Congress under the Government Performance and Results Act of

17 Personnel Development Program Performance Measures 1. The percentage of Special Education Personnel Development projects that incorporate evidence-based practices into their curricula. 2. The percentage of scholars completing Special Education Personnel Development funded programs who are knowledgeable and skilled in evidence-based practices for infants, toddlers, children and youth with disabilities. 3. The percentage of Special Education Personnel Development funded scholars who exit preparation programs prior to completion due to poor academic performance. 4. The percentage of Special Education Personnel Development funded degree/certification recipients who are working in the area(s) for which they were prepared upon program completion. 17

18 Personnel Development Program Performance Measures The percentage of Special Education Personnel Development funded degree/certification recipients who are working in the area(s) for which they were prepared upon program completion, and who are fully qualified under IDEA. 6. The percentage of Special Education Personnel Development program degree/certification recipients who maintain employment in the area(s) for which they were prepared for three or more years, and who are fully qualified under IDEA. 7. The Federal cost per fully qualified degree/certification recipient T Grants must report on Program Measures 1, 2, 3, and 5.

19 Program Performance Measures OSEP collects data on the program performance measures each year. Grantees should propose a project performance measure that addresses each program performance measure reported for the 325T program. The focus of the project measure must be the same as and clearly aligned with the program measure. 19

20 Program Performance Measure 1. The percentage of Special Education Personnel Development projects that incorporate evidence-based practices into their curricula. Example of related project performance measure: By the beginning of Year 3, 100% of syllabi from core courses will be rated as evidence-based by the Project Advisory Board using a rubric designed by the Project. 20

21 Program Performance Measure 2. The percentage of scholars completing Special Education Personnel Development funded programs who are knowledgeable and skilled in evidence-based practices for infants, toddlers, children, and youth with disabilities. Example of related project performance measure: At the end of their program, 100% of scholars will achieve a minimum of 3 on a 4-point rubric for a portfolio that provides evidence of their competencies in providing interventions to children with high incidence disabilities. 21

22 Program Performance Measure 3. The percentage of Special Education Personnel Development funded scholars who exit preparation programs prior to completion due to poor academic performance. Example of related project performance measure: As measured annually, no scholars (0%) will have exited the program due to poor academic or field-based performance. 22

23 Program Performance Measure 5. The percentage of Special Education Personnel Development funded degree/certification recipients who are working in the area(s) for which they were prepared upon program completion, and who are fully qualified under IDEA. Example of related project performance measure: As measured annually, 90% of graduates from the revised program will be working in the area for which they were trained and fully qualified under IDEA. 23

24 Types of Performance Measures Project Measures that the grantee establishes to meet their project objectives Project performance measures can address both the process of working towards an objective and the outcome related to meeting the objective Ensure a mix of both process and outcome measures, but most will be outcome Taken from the Center for Evaluation and Education Policy (CEEP) presentation at 24

25 High Quality Performance Measures High quality performance measures show What will change How much change you expect Who will achieve the change When the change will take place 25 Taken from the Center for Evaluation and Education Policy (CEEP) presentation at

26 Project Performance Measure Examples Process measure (e.g.) - Project staff (who) will hold 4 (how much) trainings with faculty on how to the use the IRIS Modules (what ) during the first and third years of the grant (when). 26

27 Project Performance Measure Examples Outcome measure (e.g.) - By the end of the third year of the grant (when), 80% of student teacher supervisors (who) will demonstrate 100% reliability (how much) when using the evidence-based practices rubric established to evaluate scholar assignments (what). 27

28 Project Performance Measure Examples Outcome measure (e.g.) - At the end of their third year of training (when), 90% (how much) of partner schools (who) will report satisfaction with the supervision (what) provided to their student teachers. 28

29 Common Problems Activities are NOT performance measures If the best response is Yes, we did that, it is likely an activity (not a performance measure) Activities: Establish a mentoring program Hold an advisory board meeting 29 Evaluate the project Taken from the Center for Evaluation and Education Policy (CEEP) presentation at

30 Common Problems Performance measures need to be measurable Examples with measurement problems (activities rather than outcomes) Will maintain collaborative partnerships with parent organizations Increase the sustainability of the personnel development program 30 Taken from the Center for Evaluation and Education Policy (CEEP) presentation at

31 Need additional information on writing performance measures? All grantees are strongly encouraged to seek training on writing performance measures. For further information on developing performance measures and logic models, see

32 Summary Projects should have a few clear objectives that explain what the project is doing to support the overall goal(s) Each objective should have a few, specific performance measures to demonstrate how progress toward meeting the objective will be measured Both program and project performance measures are included in the ED524B 32

33 Completing the ED 524B The ED 524B is a required reporting form with specific instructions. The form is used by all ED grants and has been approved by the Office of Management and Budget (OMB). Project Directors must follow the directions listed in the Dear Colleague letter and ED 524B Instructions provided by OSEP. Word or PDF versions of the forms are available at 33

34 X x 34

35 ANNUAL PERFORMANCE REPORTS Reporting Period: For first year grants, the date is the beginning of the project year to February 29, For grants in years 2-4, it is the date from the end of the previous reporting period to February 29,

36 ANNUAL PERFORMANCE REPORTS Budget Expenditures: Report the expenditures during the Reporting Period. Must be data or information from the business or grants office. 36

37 ANNUAL PERFORMANCE REPORTS Performance Measure Status: This will be checked No since OSEP is asking for data for the reporting period covering all years of the grant, not for this budget period. The date entered here will be the due date for your Final Performance Report; which is 90 days after the end of the grant. Signatory must have authority to sign on behalf of the institution since the grant is from the Department to the institution and not to an individual. The Authorized Representative signs; not the Project Director. 37

38 OMB No Exp. 02/28/2011 EXECUTIVE SUMMARY SHEET H325T *** Provide highlights of the project's activities and the extent to which the expected outcomes and performance measures were achieved during the reporting period. Do NOT include the project abstract. 38

39 PROJECT STATUS CHART H325T

40 PROJECT STATUS CHART H325T Enter one of the project s objectives; on subsequent pages, you will enter additional project objectives as submitted in your grant application. 40

41 PROJECT STATUS CHART Enter the PROGRAM performance measure(s) that align with the objective immediately followed by any PROJECT measure(s) developed to address that program measure. Enter any additional PROJECT performance measures that show you are measuring progress toward meeting the objective (e.g., 1c., 1d.,). 41

42 PROJECT STATUS CHART Here you identify if the performance measure is a PROGRAM measure, PRGM, or a PROJECT measure, PROJ. Note: Program measure refers to one of OSEP s 7 performance measures for the Professional Development Program. Project measures are unique to your grant. 42 PRGM PROJ

43 QUANTITATIVE DATA PROJECT STATUS CHART Depending on your measure, enter either a raw number, or a ratio and percentage. Enter the target number identified in the performance measure and then the actual data for this year. If complete data are not available for the measure, enter 999 (if no baseline) or NA in the Raw Number or % column, as appropriate. Provide an explanation at the bottom of the page under Explanation of Progress. 43

44 Information to Include in the Explanation of Progress Section Describe the data provided (e.g., what data collection methods were used, when were the data collected, how was a sample drawn, are there missing/incomplete data, what was the response rate, was a reliability measure taken). Your Project Officer should be able to understand and interpret the number in the chart from your description in this section. What changes in the data occurred since last APR (i.e., trend)? What activities were undertaken to achieve the targets? If targets were not met, what are possible reasons? How will activities that failed to meet targets be improved? 44

45 QUALITATIVE DATA PROJECT STATUS CHART If measure requires the collection of qualitative data, report the performance measure and type (PROG or PROJ) and then, enter N/A under the Raw Number and Percentage columns. 45 N/A N/A N/A N/A In the Explanation of Progress section of the page, referencing the performance measure by number, report applicable qualitative data along with other information about how these data were collected, targets and activities refer to previous slide for additional content requirements.

46 H Redesign curriculum to align with state licensure standards and HQT requirements. The percentage of Special Education Personnel Development projects that incorporate evidencebased practices into their curriculum. The project faculty committee will revise 100% of all syllabi in the curriculum to ensure the inclusion of evidence-based practices, by the end of the project s first year. Faculty will implement the revised syllabi with 90% fidelity documented through one observation per semester using an observation rubric. PRGM PROJ PROJ Leave boxes blank NA NA 46

47 H Redesign curriculum to align with state licensure standards and HQT requirements. Explanation of Progress (Include Qualitative Data and Data Collection Information) (1a) For this PROGRAM measure, we will submit our revised syllabi to OSEP by May 1, 2013 for inclusion in the analysis conducted by OSEP. The project faculty have revised 80% of the syllabi and expects to have revised 100% of the syllabi by the due date. Revisions were made to ensure that the areas of behavior, assessment, literacy, inclusive strategies and instructional practices are included across the curriculum, by the end of the first year of our grant. (1b) Our faculty was committed to revising our entire curriculum the end of the project s first year. Five months into the first year, we have completed a review and revision of 50% of the syllabi (10 out of 20 courses). The review process involved using an evidence-based practices (EBP) rubric developed by a faculty committee to determine whether revision was necessary to meet our new state teaching licensure standards which includes knowledge and skills on the use of EBP. We anticipate we will complete the remaining 50% by the end of the first year. (This measure links to measure 1a.) (1c) In the first five months of the project we have begun to develop an observation rubric that will be used to measure the fidelity of implementation of evidence-based practices in each of the revised courses. Currently, we have two sections of each of our 20 courses. Therefore, a total of 40 drop-in observations will be conducted each semester. We anticipate that we will complete the development of the observation rubric by the end of the first year of the project, pilot test the rubric in the first semester of Year 2, and begin full implementation of the rubric in the second semester of Year 2. We have established a target of 90% fidelity of implementation. No data are available for this reporting period as we are currently in Year 1 of our project. 47

48 Final Page of the Report H Section B: Refer to the instructions for Section B in the ED 524B Instructions Section C: Include additional information (recruitment material, syllabi, evaluation instruments, journal articles)

49 Section B Budget Information This section is never blank! A table can be helpful! A. Provide actual expenditures for this reporting period (through 2/29/2012) B. Estimate anticipated expenditures for the rest of this budget period and balance remaining, if any. C. Explain why you did not expend funds at the expected rate. D. Indicate how you plan to use the unexpended funds (carryover) in the next budget period. 49

50 Section B Budget Information E. Describe any significant changes to your budget resulting from modifications of project activities. F. Describe any changes to your budget that affect your ability to achieve your approved project activities and/or project objectives. G. Describe any anticipated changes in your budget for the next budget period that require prior approval from the Department. Any questions Talk to your Project Officer 50

51 Section C Additional Information Provide a list of current partners on your grant and indicate if: Any partners changed during the reporting period. If there were changes, please describe both the changes and any impact that resulted in your ability to achieve approved project objectives and/or project activities. Any partners are anticipated to change during the next budget period. If so, please describe both the changes and any impact the change might have on your ability to achieve approved project objectives and/or project activities. Describe any changes that you wish to make in the grant s activities for the next budget period that are consistent with the scope and objectives of your approved application. 51

52 Section C Additional Information If requesting changes to the approved Project Director and/or other key personnel, please include the person s name, title, and contact information. Indicate his/her proposed start date, and percentage of time working on the grant, and attach a resume or curriculum vitae to the annual performance report being submitted. Do not report on any key personnel changes that were already made during the current or previous budget period(s). Note: Departmental approval must be requested and received prior to making key personnel changes. Provide any other information about your project including unanticipated outcomes or benefits 52

53 Section C Additional Information Year 1 Review, Cohort IV, 2010 grantees: 325T Year One Review Status Report: Include the steps/documentation to address the suggestions of reviewers and your OSEP project officer, unless provided elsewhere in the report. Reminder: Syllabi should be sent via to the designated project officer, along with the Syllabi Revision Summary Sheet. 53

54 Submitting the ED 524B Submit the ED 524B at Instructions for using G5 are in the continuation packet. Signed ED 524B Cover Sheet must be scanned and ed in PDF format to your Project Officer. Special cases require regular submission of the 524B and signed cover sheet in PDF format to your Project Officer rather than submission through G5 Final Performance Reports or APRs for grants in their last performance period, or no-cost extension 54 Grants that have been front-loaded (forward-funded) last year

55 Contact your OSEP Project Officer with any questions! Sarah Allen Jennifer Coffey Tina Diamond Grace Zamora Duran Corinne Weidenthal Due Date: April 27,