Competency-Based Approach Leanne Worsfold September 5, 2012
Agenda Introduction to the Canadian model Define competence and competencies Review evidence-based literature Competence-based Assessments
Canadian lead case Dr. Errol Wai-Ping, a gynecologist accused of mistreating, misdiagnosing and castrating hundreds of women. The events leading to the allegation of incompetence involving the management of 47 patients between 1992 and 2001. Fifth Estate documentary http://www.cbc.ca/fifth/donoharm.html
Canadian lead case System Problems Complaints Committee referred the member to the QA Committee (Competency Assurance Panel) Individual cases were reviewed in isolation of prior history. Directed continuing education Competency assessment (peer assessment) was not concluded until well into the investigation
Changes to the legislation The case contributed to the review and revision of the Regulated Health Professions Act, 1991 S.O Continuing education and professional development alone does not assure public protection
Quality Assurance Program Regulated Heath Professions Act requires all health professions to engage in: peer and practice assessment; Self evaluation; and Continuing education and professional development. Regulatory body must develop the tools, monitor compliance and engage members in remediation.
Defining Competence Competence is more than just knowledge and skills. It involves the ability to meet complex demands, by drawing on and mobilizing psychosocial resources (including skills and attitudes) in a particular context; and Is the ability of an individual to perform a job properly knowledge, skill, judgment and attitudes.
CPSO Committee Decision Dr. Wai-Ping was in a high percentile in his residency training and despite years of experience, he exhibited knowledge gaps in basic ob/gyn areas. His inappropriate management of patients indicates a lack of knowledge and/or an inability to translate knowledge to the practice setting.
Defining Competencies A competency is a set of defined behaviors that provide a structured guide enabling the identification, evaluation and development of the behaviors in an individual; and Defines essential knowledge, skill and judgment requirements across practice areas or within a specialty that are required to provide competent, safe practice.
Defining Performance Indicators Performance indicators provide the action statement that describes how a competency is applied/demonstrated in practice.
Engages in a reflective and evaluative approach to practice that integrates findings into practice. Demonstrates insight into personal expertise and limitations. Demonstrates effective, appropriate, and timely consultation with other health professionals as needed for optimal client service. Investigates alternative explanations for deficits in occupational performance and engagement. Cues: barriers, psychosocial components, financial resources
How are Competencies Applied Provides guidance and direction to the professionals and practice settings Establishes the minimum expectation for regulation Provides a common denominator for data collection Provides the foundation for professional development and assessment tools
Competency Assessment Versus CDP Continuing Professional Development (CPD) encourages ongoing learning; Literature suggest that it does not ensure competence; and Competency assessment encourages public trust and protection.
Conscious Competence Model Consciously Competent Unconsciously Competent Consciously Incompetent Unconsciously Incompetent
Competency-based Model Competency Enhancement (Continuing Professional Development) All Members Self-assessment, professional development plan, track learning activities and reflection Prescribed education program Competency Review and Evaluation (Competency Assessment) Random Selection Portfolio review * monitors completion Two step competency assessment process
Competency Enhancement Step 1 Step 2 Step 3 Step 4 Complete your Profile Review selected standards from selfassessment and prescribed learning Track learning activities Review learning plan and determine if goal is met Complete selfassessment Develop goals and list learning activities Evaluate learning and impact on practice Review summary of continuing learning activities Complete prescribed learning Print and /or save final documents to computer
Competency Enhancement Self-assessment includes reflective practice questions Prescribed education calibrates self-assessment and supports goal development Learning module with reflective practice questions Jurisprudence test Peer feedback All items are mapped to the essential competencies
Reflection - Ascribed
Reflection - Prescribed
Competency Review and Evaluation Step 1 Random selection Portfolio review - monitor completion Multi-source feedback process Step 2 Below the established threshold In-depth practice assessment Remediation (SCERP) Self-remediation, recommendations, direction
Competency Assessment Know what you are measuring (job analysis/ competencies); Multi-facetted to include quantitative and qualitative information from multiple sources; Multi-stepped approach to ensure efficiencies and an educational process; and Assess the professional s actions in real-life situations.
Miller s Assessment Pyramid (Action) (Performance) DOES SHOWS HOW Performance assessment: client and coworker survey, Chart Review, Behaviourbased interview, direct observation Simulation/Standardize patient (Competences) (Knowledge) KNOWS HOW KNOWS Scenario Based Questions, essays Written exam Norcini, J. J BMJ 2003;326:753-755 Pg 23
Change.
One-Step Model One Step Process Face to face assessment High demand on resources (staff, time, finances / budget) High percentage competent (98.08%) Low percentage of membership is selected each year. Competency Assessment Pre-Questionnaire In-depth onsite assessment Remediation Self directed remediation Recommend ation Directed remediation
Multi-Stepped Model 5% 10% of membership is selected Step 1 Options Multi source / 360 Jurisprudence test Portfolio review Multiple choice test Chart Review Clinical Judgment Assessment Only those identified in Step 1 participate in Step 2 Step 2 Options Behaviour based interview (with chart review) Competency Assessment Step-1 Provides formative feedback to a large number of members Screening tool to identify those OTs who require more in-depth assessment Competency Assessment Step-2: Identified potential high risk group Engage in an in-depth assessment Remediation Self directed remediation Recommendation Directed remediation
Multi-Facetted Approach Pre- Questionnaire Step-1 Results Step-2 Results Review Portfolio Materials Report Written Submission Improvement Plan Impact Do
Educational
Competency Assessments Educational in nature No fail model Supports and encourages self-directed remediation Calibrates self-assessment Enhances Portfolio requirements Increasing popularity in Canada, UK, Ireland, USA
Competency-Based Tools: Benefits Formative feedback supports self reflection and changes in practice (self-remediation) Can link values / data to the member s professional development plan Tools can be implemented by practice settings Builds public trust and respect in the profession Regulatory body promotes a philosophy that is proactive and educational in nature
Leanne Worsfold Director Quality Programs and Test Development icomp Consulting Inc. 905-556-0347 lworsfold@icompconsulting.ca www.icompconsulting.com