Manitoba Practice Assessment Program: Assessing the Performance of Physicians in Practice April 2011
Conflict of Interest The Manitoba Practice Assessment Program (MPAP) conforms to the rules, regulations, policies and procedures of the University of Manitoba, Faculty of Medicine, and the College of Physicians and Surgeons of Manitoba.
Objectives Identify the various components of the MPAP. Describe the results of the CanMEDS weighting research project. Discuss strategies and practices being used to assess the competence and performance of physicians and healthcare professionals.
MPAP Team Dr. Jose Francois MD, Associate Dean Dr. Marilyn Singer MD, Director Dr. Jillian Horton MD, Assistant Director Dr. Brenda Stutsky PhD, Program Advisor Debbie Lemkey, Program Assistant
Goal To offer an assessment process that is: Comprehensive Transparent Tailored to the physician s specialty and practice setting
Eligibility Determined by the College of Physicians and Surgeons of Manitoba (CPSM). The CPSM has the authority to deem any physician ineligible.
Eligible Physicians Attempted either Part 1 and/or Part 2 of the Medical Council of Canada Qualifying Exam (MCCQE); and/or Attempted to become certified by the CFPC or the RCPSC; and Currently practicing in Manitoba on the Conditional Register; and Been in active practice in Manitoba for a period of 2 years (Review of self-assessment must determine that sufficient data exist to complete an assessment).
Process Starts with a referral from the CPSM Ends with: Full report sent to the candidate. Period of 10 days to appeal based on procedural error. Letter sent to CPSM with access to the full report if requested.
Framework
Assessment Components Self-Assessment Self-Assessment Report Self-Assessment: Reflective Practice (Matches 360 Questions) Self-Assessment: Clinical Practice Skills 360 Degree Surveys Physician Colleagues Interprofessional Colleagues Patients Chart Audit/Chart Stimulated Recall Observation of Clinical Practice Key Interviews Comprehensive Report
Technology Forms and information management system is in development. Web-based. All assessment tools incorporated into the system. Auto generate a report. Image from Creative Commons http://www.flickr.com/photos/johncatral/4537877423/sizes/m/in/photostream/
Research Questions: 1. How do physicians rate the complexity, frequency, and criticality of CanMEDS roles? 2. How do the CanMEDS roles compare with one another in terms of their relative levels of perceived importance? CanMEDS Weighting Project Image from Creative Commons http://www.flickr.com/photos/davehamster/531768924/sizes/m/in/photostream/
2 Online Surveys
Sample Physicians on Scott s Directories: Canadian Medical Directory BC, AB, SK, MB, NWT, Yukon, & Nunavut Final sample size N=88 (23 family practice, 65 specialists) Image from Creative Commons http://www.flickr.com/photos/mega1210/4948863429/sizes/m/in/photostream/
Analysis Index of Practice Importance Importance = Frequency x Criticality Activity Weight Weight = Sum of Mean Importance/Number of Competencies (for each of the 7 CanMEDS roles) References Kane, M. T., Kingsbury, C., Colton, D., Estes, C. (1989). Combining data on criticality and frequency in developing test plans for licensure and certification examinations. Journal of Educational Measurement, 26(1), 17-27. Raymond, M. R. (2001). Job analysis and the specification of content for licensure and certification examinations. Applied Measurement in Education, 14(4), 369-415.
Results No significant difference between family practice and specialists in index of practice importance scores. Mean complexity scores: Medical Expert 4.36 Collaborator 4.04 Communicator 4.03 Health Advocate 3.99 Manager 3.96 Professional 3.85
CanMEDS Weighting
www.umanitoba.ca/cpd