What does accreditation do for quality in medical education? The evidence and the origins of ECFMG policy

Transcription

1 What does accreditation do for quality in medical education? The evidence and the origins of ECFMG policy Marta van Zanten, Ph.D. Foundation for Advancement of International Medical Education and Research (FAIMER) Educational Commission for Foreign Medical Graduates (ECFMG)

2 Overview of presentation Description and rationale of ECFMG 2023 accreditation policy Accreditation research

3 Current ECFMG Certification Requirements for Graduates of International Medical Schools Seeking PME in the United States Medical Education Credentials School and graduation year listed as current in the International Medical Education Directory (IMED) School recognized to grant the MD degree (or equivalent) by local government USMLE Examinations

4 New ECFMG Certification Requirement In addition to the current requirements: Effective in 2023, ECFMG applicants will be required to graduate from an appropriately accredited medical school Recognition of the accrediting agency by the World Federation for Medical Education (WFME) will fulfill this new 2023 requirement

5 Rationale for New ECFMG Requirement ECFMG mission: protect the public and actively seeks opportunities to promote medical education through programmatic and research activities

6 Rationale for New ECFMG Requirement Protect the public ECFMG Certification focuses mostly on evaluating the individual USMLE Medical school requirement: School must be recognized Permission to grant degree, vs. Legitimacy of credentials for licensure One-time permit to open, vs. Ongoing process of evaluation New accreditation policy will help ensure protection of the public by better ensuring quality of medical schools

7 Rationale for New ECFMG Requirement Actively seeks opportunities to promote medical education through programmatic and research activities Accreditation helps ensure quality medical education programs, leading to high quality graduates and the provision of enhanced health care Accreditation fosters ongoing institutional and program improvement

8 Rationale for New ECFMG Requirement Accreditation can also vary Process Complexity Transparency Accountability Consequences New ECFMG accreditation policy will help catalyze efforts around the world to develop system using globally accepted criteria enhance harmonization of accreditation standards

9 Accreditation Research van Zanten M, Boulet JR. The association between medical education accreditation and examination performance of internationally educated physicians seeking certification in the United States. Quality in Higher Education. 2013;19(3):

10 Background Approximately 2/3 of countries with medical schools have a system of accreditation Limited data linking accreditation to better performance of students / graduates Examine medical education accreditation practices worldwide Special focus on the Caribbean Determine the association between the existence of a national system of accreditation and USMLE performance of graduates who seek ECFMG certification

11 Background ECFMG certifies approximately 10,000 IMGs per year IMGs attended medical schools in countries all over the world 26% attended a school located in the Caribbean region Other 46% India 19% Dominica 8% Grenada 7% Pakistan 6% Cayman Islands 2% Egypt 2% Israel 2% China 3% Philippines 3% Sint Maarten 2%

12 Methods Population All IMGs taking one or more USMLE exams during the study period ( ) Attended a medical school located in a country with a mandatory system of accreditation, or a voluntary system and all schools are accredited (accred=yes) OR Attended a medical school located in a country without a system of accreditation (accred=no) Countries with a voluntary system resulting in some schools accredited and some not accredited were eliminated Countries with fewer than 10 takers for each exam were eliminated

13 Methods Person-level variables Gender (male or female) Years elapsed since graduation at time of testing (< 3 years or 3 years) Self-reported native language (English or all others) Accreditation status of the IMGs country of medical education (yes or no)

14 Analysis First attempt pass rates on all USMLE exams (Step 1, Step 2 CK and Step 2 CS) were calculated based on person level variables and on accreditation status of medical school Separately for each exam, a generalized linear model was used to investigate the impact of accreditation after controlling for person level variables Analyses were conducted separately on data from Caribbean and all other (non-caribbean) IMGs

15 Results 173 countries/territories with medical schools listed in IMED 118 countries met inclusion criteria 19 Caribbean countries 99 Non-Caribbean countries IMGs who took an exam for the first time during the study period Step 1: 67,000 Step 2 CK: 55,000 Step 2 CS: 58,000

16 Results (Step 1) Non-Caribbean Caribbean 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0%

17 Results (Step 2 Clinical Knowledge) Non-Caribbean Caribbean 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0%

18 Results (Step 2 Clinical Skills) Non-Caribbean Caribbean 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0%

19 Accredited Nonaccredited Accredited Nonaccredited Accredited Nonaccredited Accredited Non-Accredited Accredited Non-Accredited Accredited Non-Accredited Results (Accreditation) 100% Non-Caribbean 100% Caribbean 90% 90% 80% 80% 70% 70% 60% 60% 50% 50% 40% 40% 30% 30% 20% 20% 10% 10% 0% 0% Step 1 Step 2 CK Step 2 CS Step 1 Step 2 CK Step 2 CS Adjusted ORs

20 Discussion The existence of a system of accreditation had the largest positive association with Step 1 performance Many accreditation criteria relate to basic science phase of curriculum Students experiences in clinical phase are more varied and less influenced by accreditation Association between accreditation and performance was greatest in the Caribbean Important finding due to the large numbers of Caribbean graduates seeking to practice in the United States Higher quality schools may be more likely to seek voluntary accreditation, and attract higher quality applicants

21 Limitations Physicians who take USMLE and seek ECFMG certification are self-selected and may not be representative of entire student population No pre-matriculate performance variables were available USMLE performance is one outcome, may not be best predictor of physician quality Accreditation status (yes/no) does not reflect robustness of accreditation process Cross-sectional study, no support for causation

22 Conclusions / Future Studies This study lends support to the value of accreditation in some regions Links existence of accreditation systems to better performance of students/graduates Future studies should include longitudinal research Compare performance of students pre- and post- accreditation activities Investigate association between accreditation and other outcome measures Investigate added value of various accreditation processes

23 van Zanten M, Boulet JR. The association between medical education accreditation and examination performance of internationally educated physicians seeking certification in the United States. Quality in Higher Education. 2013;19(3): van Zanten M, McKinley D, Durante Montiel I, Pijano CV. Medical education accreditation in Mexico and the Philippines: impact on student outcomes. Medical Education. 2012;46(6):

24 Thank you Marta van Zanten