Next Accreditation System II. Annual Program Review

Transcription

1 Accreditation Council for Graduate Medical Education Next Accreditation System II. Annual Program Review Pamela L. Derstine, PhD, MHPE RRC Executive Director SNS Program Director Workshop June 8, 2013

2 Topics Annual Data Submission Screening Annual Data Review of Annual Data

3 Annual RRC Program Review Annual ADS Update Program Changes Structure and resources Program Attrition PD / core faculty / residents Scholarly Activity Faculty and residents Board Pass Rate 7 year rolling average Clinical Experience Case logs Resident Survey Common and specialty elements Faculty Survey Semi-Annual Resident Evaluation and Feedback Milestones Omission of data

4 Annual RRC Program Review Annual ADS Update Program Changes Structure and resources Participating Site information (e.g., PLA on file, distance from primary site, rationale, rotation months) Current Block Schedule: programs must follow instructions and use the block schedule template provided agraminstructions.pdf Sponsoring institution information (e.g., DIO, participating sites, other sponsored programs)

5 Annual RRC Program Review Annual ADS Update Program Changes Structure and resources Major changes Response to current citations (if resolved on annual review, will be removed and stored in program history) Duty Hour, Patient Safety, Learning Environment (17 items) Overall evaluation methods (6 items)

6 Annual RRC Program Review Annual ADS Update Program Attrition PD / core faculty / residents Program director change history since 2000 Program director CV Program faculty basic information Resident basic information Change requests

7 Annual RRC Program Review Annual ADS Update Scholarly Activity Faculty and residents CV used for program director Scholarly activity templates used for faculty and residents Scholarly activity is reported for the most recently completed academic year (12 month period, not a 5 year period) for all but the PD

8 Program Director CV same information requested but entered online

9 Scholarly Activity Template

10 Faculty Scholarly Activity Enter Pub Med ID # s

11 Faculty Scholarly Activity Enter a number 2013 Accreditation Council for Graduate Medical Education (ACGME)

12 Faculty Scholarly Activity Enter a number 2013 Accreditation Council for Graduate Medical Education (ACGME)

13 Faculty Scholarly Activity Enter a number 2013 Accreditation Council for Graduate Medical Education (ACGME)

14 Faculty Scholarly Activity Enter a number 2013 Accreditation Council for Graduate Medical Education (ACGME)

15 Faculty Scholarly Activity Answer Yes or No 2013 Accreditation Council for Graduate Medical Education (ACGME)

16 Faculty Scholarly Activity Answer Yes or No 2013 Accreditation Council for Graduate Medical Education (ACGME)

17 Faculty Scholarly Activity 2013 Accreditation Council for Graduate Medical Education (ACGME)

18 Resident Scholarly Activity

19 Resident Scholarly Activity

20 Resident Scholarly Activity

21 Annual RRC Program Review Board Pass Rate: 5 year ADS self-report

22 Annual RRC Program Review Board Pass Rate 7 year rolling average ABNS has provided pass rates to the ACGME electronically for each year beginning with 2003 through 2012 for parts 1 and 2 for all programs ABNS will provide an annual electronic update to the ACGME, beginning with the 2013 exam results Annual ABNS reports to ACGME may preclude the need for programs to provide this information

23 Annual RRC Program Review Clinical Experience Case logs program review - minimum numbers for defined case categories reviewed and feedback provided (no citations based on min. numbers) ; program reviews - graduates expected to comply with minimum number requirements for all categories except critical care (DC20-28) and endovascular (DC3b) - level-specific reports will be reviewed to monitor progress towards compliance program review and beyond - graduates expected to comply with all categories

24 Annual RRC Program Review Resident Survey Common and specialty elements 7 survey question domains: duty hours; faculty; evaluation; educational content; resources; patient safety; teamwork 70% response rate required Aggregated non-compliant survey responses for each domain are reviewed; thresholds for noncompliance Exception: programs with DHE and non-compliant responses for intimidation and service-overeducation will be cited

25 Annual RRC Program Review Resident Survey Common and specialty elements Programs with DHE and non-compliant responses to 88-hour question - first time non-compliance: warning - second time non-compliance: action plan required - third time non-compliance: DHE canceled; program may not reapply for DHE

26 Annual RRC Program Review Faculty Survey 5 question domains: - supervision and teaching - educational content - resources - patient safety - teamwork Intended to mirror most resident survey questions and provide opportunity to compare responses by question domain First survey completed: spring 2013 First RRC review of faculty survey data: spring 2014 Compliance metrics not yet developed

27 Annual RRC Program Review Milestones First milestone evaluation period: July December Residents evaluated as usual by the program (competency-based, multiple evaluators) First milestone reports to ACGME: Nov/Dec Collected evaluations reviewed by the CCC - CCC determines milestone level for each resident for each milestone - Milestone reporting will be done through a link in ADS (not yet available) Second milestone reports to ACGME: May/June 2014 First RRC review of milestone data: spring 2015

28 Annual RRC Program Review Annual ADS Update Omission of data If any required annual ADS update information is missing, the program will be flagged by the NAS data system Data omission could result in an altered accreditation status

29 Screening Annual Data Thresholds for core annual data established to quickly identify programs with serious problems Aids in prioritizing Review Committee work Facilitates more timely identification of problems and provision of feedback to programs to correct problems Goal ensure ongoing quality resident education

30 Screening for Program Changes Three or more changes (e.g., participating site, resident complement, block diagram, major structural change, sponsoring institution change, GMEC report of structure change)

31 Screening for Program Attrition Three or more program attrition reports (e.g., program director change, core faculty decrease, chair change. DIO change, CEO change, resident attrition)

32 Screening for Scholarly Activity Faculty < 100 % faculty with at least two points per year Activity in each category counts as one point except Each peer-reviewed article and grant counts as one point Residents < 80% of residents with at least one point per year Activity in each category counts as one point except Each peer-reviewed article counts as one point

33 Screening for Board Scores < 85% first time takers (for credit) pass primary exam averaged over last three years < 80% first time takers pass oral exam (minimum of 10 residents)

34 Screening for Case Logs Each graduating resident: below minimum number in any defined case category reporting < 500 total cases (not limited to defined case categories)

35 Screening for Surveys Resident Non-compliance in duty hours area Non-compliance in three or more areas Faculty Not yet established

36 Screening for Milestones Not yet established

37 Screening for Omissions One or more core data elements omitted

38 Policies and Procedures Policies and Procedures: 7/1/ cedures.pdf Policy Review of Annual Data Data RRC may use RRC actions following annual data review Policy Accreditation Actions

39 Review of Annual Data Continuous data collection/review ADS annual update Resident survey Faculty survey Milestone data Certification examination performance Case log data CLER information Hospital accreditation data Faculty and resident scholarly activity / productivity Other

40 Review of Annual Data Other data (episodic) ACGME complaints (section 23.10) Verified public information Historical accreditation decisions / citations Institutional quality and safety metrics Other

41 Accreditation Decisions NO proposed adverse actions Potential Actions (if currently accredited): progress report focused site visit full site visit continued accreditation accreditation with warning (may not request permanent complement increase or DHE)

42 Accreditation Decisions Potential Actions (if currently accredited): probation (must be preceded by a site visit; status limit of 2 years; may not request permanent complement increase or DHE) complement reduction withdrawal of accreditation

43 Accreditation Decisions Potential Actions (if currently accredited): Recognize exemplary performance; innovations Identify opportunities for program improvement Identify concerning trends Issue new citations Continue previous citations Acknowledge correction of previous citations

44 Accreditation Decisions Accreditation decisions made in the past 12 months will be updated effective July 1, 2013 Short-cycled programs (1-2 year cycle lengths) will change from continued accreditation to accreditation with warning Accreditation with warning is NOT an adverse action; residents and applicants do NOT need to be informed