Nicholas W. Gabriel, CRNA, PhD
Objectives The Study Problem Literature Review and Conceptual Framework Methodology Results Discussion
The Study Problem IOM Reports Competency of Nurse Anesthetists Recertification Written Examination Self-assessments Performance assessment Simulation
Purpose of Study Relationships between written exam scores, selfassessments scores, and performance assessment scores in a simulated environment of nurse anesthetists
Literature Review Results from 13 studies comparing written examination scores and performance scores found all (21) correlations to be positive; those with higher scores on examinations had high performance scores, although only 10 of these were reported to be statistically significant. The correlations ranged between 0.09 and 0.54; majority of correlations were under 0.20.
Literature Review Murray, Henrichs, et. al 2007: developed 12 scenario assessment instrument for anesthesia residents and providers Henrichs, et al. 2009: first use of assessment instrument on nurse anesthetists Mudumbai et. al, 2012 hybrid study using six of the scenarios on graduating 3rd year anesthesiology residents
Miller s Model of Competence Figure 2 Miller's pyramid of competence SP=simulated patients; OSCE=objective structured clinical examination; HCQ=multiplechoice questions.
Methodology Research Design Sample Data Collection Methods Procedure Data Analysis
Sample Practicing CRNA s in the Sacramento area Total of 143 CRNA s in Sacramento area 18 participants volunteered for study
Data Collection Methods Measurement Tools Demographic data Self-assessment Written knowledge examination Performance assessment tool
Written Knowledge Examination 30 question multiple choice examination Four core competencies Airway Pharmacology Physiology and Pathophysiology Technology
Procedure Recruitment Simulation Center Preparation Informed consent and demographic data Pre-test self-assessment Written Examination Performance Assessment in Simulation Lab Post-test self-assessment
Data Analysis Demographic data: age, years of experience, practice setting, prior exposure to simulation Data analyzed to detect any systematic differences
G-Theory Used to estimate reliability and measurement fidelity of instruments Provide estimates of variance between sources Person X rater X scenario
Results Demographic Data Mean Age 43.90 (30-64) Workplace: Kaiser 5 Workplace: UCDMC 13 Mean Years of Experience 8.5 (2-20) Prior Exposure to Simulation Yes(No) 72% (28%)
Figure 1. Percentage of Key Action Scores Accomplished Malignant Hyperthermia Oxygen Pipeline Failure Total Spinal Simulation Performance Scenario Tension Pneumothorax Right Main Stem Intubation Hyperkalemia Bronchospasm Acute Hemorrhage 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 Total Score
Correlations None among written exam and performance scores None among self-assessment scores and performance scores None among written exam and self-assessment scores
Correlations Workplace: no correlations Age and years of experience: no effect on workplace and simulation scores No difference in performance scores between Kaiser and UCDMC Prior exposure to simulation: no correlations
Written Examination Limitations: Test length and content validity No studies found correlating recertification scores with performance assessment
Limitations Number of participants 12.6% (18/143) Number of scenarios? One video camera used Equipment issues: anesthesia machine, mannequin breath sounds Validity of scenarios/key action scores Debrief not used
Implications Education Policy and Practice Research
Education new paradigm of life-long learning Changes in recertification: written examinations, various methods of assessing competency
Policy and Practice Defining competence and performance for each discipline and specialty Relation to improved patient safety and patient outcomes New recertification requirements: dynamic and evolving
Percentage of Key Action Scores Accomplished Malignant Hyperthermia Oxygen Pipeline Failure Total Spinal Simulation Performance Scenario Tension Pneumothorax Right Main Stem Intubation Hyperkalemia Bronchospasm Acute Hemorrhage 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 Total Score