I do not have an affiliation (financial or otherwise) with a pharmaceutical, medical device or communications organization.

Size: px

Start display at page:

Download "I do not have an affiliation (financial or otherwise) with a pharmaceutical, medical device or communications organization."

Austen Cannon
7 years ago
Views:

1 Selection for Family Medicine Residency training: is it time to sharpen our tools? A study of the consistency of selection processes across Programs in Canada. Keith Wycliffe-Jones BSc(Med Sci) MBChB FRCGP CCFP University of Calgary Shirley Schipper MD CCFP University of Alberta Maureen Topps MB ChB CCFP FCFP PBDM University of Calgary Jeanine Robinson BA MPA University of Calgary Kent G.Hecker PhD University of Calgary Alianu Kingsly Akawung MSc Applied Statistics, BSc Biomed University of Calgary ICRE October 2015

2 I do not have an affiliation (financial or otherwise) with a pharmaceutical, medical device or communications organization. Je n ai aucune affiliation (financière ou autre) avec une entreprise pharmaceutique, un fabricant d appareils médicaux ou un cabinet de communication. Author: Keith Wycliffe-Jones Date: October 25 th 2015

3 Objective Present initial results that address the question: Do Family Medicine Residency Programs rank the same applicants consistently?

4 Background 2013 Canadian Residency Matching Service(CaRMS) 1 st iteration 2695 Canadian Medical Graduate participants applied to Family Medicine(FM) for 1268 positions Canada - 17 schools; 106 sites 979 applicants put FM as 1 st choice UK experience - General Practice selection - no level of consistency (kappa coefficient -0.3) in ranking by different Deaneries 2 1.Canadian Resident Matching Service(CaRMS). (accessed April 23 rd 2014) 2.Plint,S. A study of the outcomes of simultaneous multiple applications to different deaneries in the UK for vocational training in general practice - is there a case for standardization of selection and processes. Dundee; 2005.

5 Background High stakes assessment process Canadian FM Programs more heterogeneous than UK Expected outcome - Family Physician who can practice anywhere in Canada. Organizational Justice 3,4,5 i) Distributive Justice - do applicants receive the selection outcomes they deserve? ii) Procedural Justice - fairness of procedure itself 3.Gilliband SW. The perceived fairness of selection systems: an organizational justice perspective. Academy of Management Review 1993; 18: Gilliland SW. Effects of procedural and distributive justice on reactions to a selection system. Journal of applied psychology 1994; 79(5): Patterson F, Zibarras L, Carr V, Irish B, Gregory S. Evaluating candidate reactions to selection practices using organizational justice theory. Med Educ 2011 Mar; 45(3):

6 Background Distributive Justice Equity applicants should receive rewards that are consistent with their inputs (self-conceptions of ability or qualifications for job), relative to a referent comparison. Equality - equal opportunity to apply Needs - rewards distributed based on individual needs. Gilliland SW. The perceived fairness of selection systems: an organizational justice perspective. Academy of Management Review 1993; 18: Gilliland SW. Effects of procedural and distributive justice on reactions to a selection system. Journal of applied psychology 1994; 79(5): Patterson F, Zibarras L, Carr V, Irish B, Gregory S. Evaluating candidate reactions to selection practices using organizational justice theory. Med Educ 2011 Mar; 45(3):

7 Hypothesis (based on distributive justice) There will be a high-degree of consistency in how CMG s who apply to more than one FM Residency Program are ranked particularly those Programs/Sites seeking similar attributes

8 Methodology Program ranking data for all FM Programs/Sites Student ranking data Multi level modelling All students are ranked by some Programs (all to which they have applied) Programs are nested within students interested in the variation in school rank within student Calculate intra-class correlation coefficient (ICC) as a measure of consistency in ranking of students across All Programs and Sites The Ottawa Conference 2014

9 Analysis - example grid of Program rank order data Program A Program B Program C Program D Program E Student ID ID # ID # ID # ID # ID #

10 Analysis - Preliminary results 1. Across all 17 FM Residency Programs (2013) Within student variability = (high) ICC (intra class correlation) = 0.22 (low) low consistency of ranking of the same students between Programs

11 Analysis - Preliminary results Selection for FM Residency Training 2. ICC - all 17 FM Residency Schools ( ) ICC Year ICC

12 Analysis-sub-sets Preliminary results 3. Subset analyses a) 4 rural FM Programs/Sites in same Western Province ICC = 0.66 b) 15 Western Canada rural FM Programs/Sites ICC = 0.34 c) 2 urban FM Programs in same Western Province ICC = 0.02

13 Analysis-sub-sets Preliminary results Subset analyses(contd.) d) 19 Eastern Canada rural FM Programs/Sites -ICC = 0.50 e) 17 larger urban-based* Programs/Sites -ICC= 0.19 (*>100,000 population, urban-based programs/sites)

14 Summary of preliminary results Suggest a very low to low level of consistency in ranking of the same students between programs/sites Evidence of higher level of consistency in the ranking of the same students between rural programs/sites

15 Questions? Keith Wycliffe-Jones

16 Help us improve. Your input matters. Download the ICRE App, Visit the evaluation area in Pre-function Hall B, near Registration, or Go to: icreevaluations to complete the session evaluation. Aidons-nous à nous améliorer. Votre opinion compte. Téléchargez l application de la CIFR Visitez la zone d évaluation, au vestibule de la salle B, près du kiosque d inscription, ou Visitez le afin de remplir une évaluation de la séance. You could be entered to win 1 of 3 $100 gift cards. Vous courrez la chance de gagner l un des trois chèques-cadeaux d une valeur de 100 $.

Optimization of clinical teaching feedback delivery in radiology residency. Colin Mar, MD Silvia Chang, MD Bruce Forster, MD

Optimization of clinical teaching feedback delivery in radiology residency Colin Mar, MD Silvia Chang, MD Bruce Forster, MD I do not have an affiliation (financial or otherwise) with a pharmaceutical,