Effectiveness of Simulation-Based Education in Critical Care Nurses Continuing Education: A Systematic Review

Clinical Simulation in Nursing (2012) -, e1-e6 www.elsevier.com/locate/ecsn Featured Article Effectiveness of Simulation-Based Education in Critical Care Nurses Continuing Education: A Systematic Review Miia Jansson, PhD-c, MNS*, Maria K a ari ainen, PhD, Helvi Kyng as, PhD Institute of Health Science, University of Oulu, Finland KEYWORDS critical care; effectiveness; knowledge; nursing education; simulation; skills; staff development Abstract Background: The effectiveness of simulation-based education in nurses continuing education is still largely unknown. The aim of this review was to assess the current body of literature regarding the effectiveness of simulation-based education in critical care nurses continuing education, with a focus on knowledge and skills. Method: We reviewed studies published between 2002 and 2011 contained in eight multidisciplinary databases (Ovid MEDLINE Ò, the Cumulative Index to Nursing and Allied Health Literature, Cochrane Library, Scopus, Web of Science, the Educational Resources Information Center, PsycINFO, and Academic Search Premier). Two researchers selected the studies and independently assessed their quality. Results: Because of a lack of available studies, only one study is included in the review. According to the results of that study, simulation-based education increases critical care nurses adherence to recommendations about safe medication administration, and thus it improves patient safety and the quality of care. Conclusion: The effect of simulation-based education on critical care nurses knowledge and skills is still questionable because of a lack of published studies and robust evidence. Further multicenter, randomized controlled trial follow-up studies are needed to assess the effectiveness of simulation-based education versus other educational interventions, as well as to develop a universal method for measuring the quality of outcomes. Cite this article: Jansson, M., K a ari ainen, M., & Kyng as, H. (2012, -). Effectiveness of simulation-based education in critical care nurses continuing education: A systematic review. Clinical Simulation in Nursing, Vol(X), e1-e6. http://dx.doi.org/10.1016/j.ecns.2012.07.003. Ó 2012 International Nursing Association for Clinical Simulation and Learning. Published by Elsevier Inc. All rights reserved. Introduction Simulation-based education is an innovative (Lupien, 2007) and interactive teaching strategy (Gaba, 2004). The effectiveness of simulation-based education in continuing education is still largely unknown despite its use in health care * Corresponding author: miiajans@mail.student.oulu.fi (M. Jansson). providers education for more than 50 years. However, the past 10 years have seen a rapid increase in awareness and adoption of simulation-based education to improve the safety culture as well as health care providers knowledge and skills, including nontechnical skills, such as multiprofessional teamwork, situational awareness, decision making (Lupien, 2007), critical thinking, and selfconfidence (Jeffries, 2005). 1876-1399/$ - see front matter Ó 2012 International Nursing Association for Clinical Simulation and Learning. Published by Elsevier Inc. All rights reserved. http://dx.doi.org/10.1016/j.ecns.2012.07.003

e2 According to the European Council and the World Medical Association s Declaration of Helsinki, health care providers education plays a key role in improving patient safety (Mellin-Olsen, Staender, Whitaker, & Smith, 2010), and it should be introduced at all levels within the health care system (Rall, Key Points Simulation-based education increases critical care nurses adherence to recommendations about safe medication administration, and thus improves patient safety and the quality of care. The effect of simulation-based education on critical care nurses knowledge and skills is still questionable because of a lack of published studies and robust evidence. Further multicenter, randomized controlled trial follow-up studies are needed to assess the effectiveness of simulation-based education as well as to develop a universal method for measuring the quality of outcomes. van Gessel, & Staender, 2011). The dynamic nature of critical-care settings requires new and more effective evidence-based teaching methods to ensure patient safety and high quality care (Cronenwett, Sherwood, Barnsteiner, Disch, Johnson, & Mitchell, 2007; Simerka, 2009). Blackwood, Albarran, and Latour (2010) have identified, prioritized, and developed a research agenda for critical care nursing in Europe. The consensus of the expert panel was that education and the effectiveness of continuing education in improving the competencies of nurses were considered important, whereas a lack of knowledge and skills jeopardized patient safety and thus the quality of care (Blackwood et al., 2010). In recent years, a number of systematic reviews have been undertaken to investigate the use of simulation-based education for undergraduate nursing students (Cant & Cooper, 2010; Lapkin, Levett-Jones, Bellchambers, & Fernandez, 2010), medical students (McGaghie, Issenberg, Petrusa, & Scalese, 2010) and interprofessional (Zhang, Thompson, & Miller, 2011) education. However, there is currently a lack of knowledge about the effectiveness of simulation-based education in critical care nurses continuing education. The aim of this study was to address this need by performing a systematic review of the current body of literature regarding the effectiveness of simulation-based education in critical care nurses continuing education, with a focus on knowledge and skills. The main question addressed in the study was, What is the effectiveness of simulation-based education on critical care nurses knowledge and skills? Only intervention studies focusing on high-fidelity or full-scale simulations, used with or without other educational strategies, were included in the review. Qualitative studies were excluded from the review. Method Search Strategy The systematic review was conducted in the fall of 2011 according to the Centre of Reviews and Dissemination guidelines (Centre for Reviews and Dissemination, 2009). A limited review was initially conducted in the Cumulative Index to Nursing and Allied Health Literature (CINAHL) and in Ovid MEDLINE Ò to help identify the optimal search terms (Table 1, online extra). The final review focused on peer-reviewed empirical studies written in English, Swedish, or Finnish (because of a lack of resources and facilities for translation of other languages) and published during the past 10 years (2002e2011). The review was conducted in eight multidisciplinary databases: Ovid MEDLINE Ò,CI- NAHL, Cochrane Library, Scopus, Web of Science, the Educational Resources Information Center (ERIC), PsycINFO, and Academic Search Premier (Table 2). A comprehensive literature search strategy was employed and carefully documented (Centre for Reviews and Dissemination, 2009). It involved comparisons of subject heading, advanced and basic, keywords, index- and MeSH terms searches with appropriate permutations, done in association with a library information specialist (Centre for Reviews and Dissemination, 2009; McGowan & Samson, 2005). The RefWorks Ò Web-based research management tool was used to maintain original searches, and the search process was documented carefully (Centre for Reviews and Dissemination, 2009). Inclusion Criteria and Study Selection Studies were included if they met the inclusion criteria (Table 3, online only) based on the research questions and the target population (registered critical care nurses), intervention (simulation-based education), type of outcomes (knowledge and skills), and study design (intervention study). The exclusion criteria are shown in Figure 1. The study selection process was carried out by two researchers independently and objectively (Centre for Reviews and Dissemination, 2009). Study selection was Table 2 Databases and Number of Original Studies Database Number of Original Studies Ovid MEDLINE Ò 19 CINAHL 22 Cochrane 3 Scopus 42 Web of Science 23 ERIC 3 PsycINFO 7 Academic Search Premier 22 CINAHL ¼ Cumulative Index to Nursing and Allied Health Literature; ERIC ¼ Educational Resources Information Center.

e3 Exclusion criteria Title (n = 71) - ICU, neonatal, pediatric (n = 7) - Other education (n = 2) - Nontechnical skills (n = 6) - Practical nurse, physician, student (n = 6) - Qualitative research, surveys, review (n = 2) Abstract (n = 48) Exclusion criteria Full text (n = 8) - Other education (n = 4) - Nontechnical skills (n = 1) - Practical nurse, physician, student (n = 14) - Qualitative research, surveys, review (n = 21) Exclusion criteria Included study (n = 1) - Conference abstract (n = 2) - Qualitative research, surveys, review (n = 4) - Nontechnical skills (n = 1) Quality appraisal (n = 1) Ford, Seybert, Smithburger, Kubulinsky, Samosky, & Kane-Gil, 2010. Figure 1 Flow chart of the study-selection process. ICU ¼ intensive care unit. performed in three stages to minimize the risk of errors and bias and ensure that all relevant studies were included. In the first stage (N ¼ 148), duplicate publications (n ¼ 62) within the eight different databases and publications from 2001 and earlier (n ¼ 15) was excluded from the review in order to reduce publication bias (Centre for Reviews and Dissemination, 2009). In the second stage, potentially relevant studies (n ¼ 71) were assessed independently by two reviewers, who compared the titles (n ¼ 71) and abstracts (n ¼ 48) against the predetermined inclusion criteria (Centre for Reviews and Dissemination, 2009). In the third stage, the full texts (n ¼ 8) of studies that appeared to meet the inclusion criteria were obtained for detailed assessment against the inclusion criteria (Centre for Reviews and Dissemination, 2009). There was complete agreement between the reviewers final selections. The study selection process was documented clearly (Figure 1) to ensure reproducibility (Centre for Reviews and Dissemination, 2009). Two of the eight studies were conference abstracts, and the full texts were not available. Quality Assessment The widely used standardized Joanna Briggs Institute Critical Appraisal Checklist for Cohort/Case Control Appraisal (Joanna Briggs Institute, 2008) was used to assess the quality of the studies (Table 4, online extra). Relevant studies (Figure 1) were assessed for methodological validity by two reviewers independently prior to inclusion in the review. To quantify quality, the reviewers assigned scores of 0 or 1. One point was assigned if the item was expressed in the study. Zero points were given if the item was not expressed or it was unclear whether it was expressed in the study. The total possible quality score ranged from 0 to 9. There was complete agreement between the reviewers in this respect.

e4 Table 5 Study, Country Ford, Seybert, Smithburger, Kubulinsky, Samosky, & Kane-Gil, 2010, USA Original Study Included in the Review Method, Participants, Setting A single-center, parallel, controlled, prospective study 24 nurses, 2 adults, ICUs direct observation, quiz Intervention Traditional didactic lecture vs. simulation-based training with human patient simulator (SimMan Ò ) Outcome Measures Medication error rates, Knowledge Results After simulation-based educational intervention, medication administration error rates decreased from 30.8% to 4% Mean quiz scores were significantly improved after education sessions in both intervention groups. Data Extraction and Analysis The extracted data included details about the participants, data collection, and data analysis, as well as specific details about controlling confounding factors and other biases (Table 5). Further analysis was limited because of the lack of available data. Results Because of the lack of available studies, only one study was included in the final review (Table 5). The quality of the study by Ford et al. (2010) was assessed as high (7 out of 9 points). The study demonstrated significant advantages of simulation-based education in improving patient safety through the reduction of medication administration error rates (Ford et al., 2010). The included article described a single-center, parallel, controlled, prospective study conducted in adult critical care and intensive care units. The primary purpose of the study was to compare medication administration error rates and assess knowledge of material before and after educational sessions conducted as either a traditional didactic lecture or a simulation-based educational intervention (Ford et al., 2010). The Effectiveness of Simulation-Based Education on Medication Administration Error Rates In the included study, 24 critical care nurses were observed administering medications during several 4-hour sessions. Data collection comprised baseline observations, initial postintervention observations (1e4 weeks after the intervention), and final postintervention observations (8e12 weeks after the intervention). Errors were classified according to 14 categories consistent with the United States Pharmacopeia MEDMARX system. After simulation-based educational intervention, medication administration error rates decreased from 30.8% to 4.0% in the initial postintervention observation (p <.001) and continued to be decreased at 6.2% in the final postintervention observation (p <.001). After the didactic lecture, medication administration error rates increased in the initial postintervention observation from 20.8% to 22.7% (p ¼.672) and increased significantly to 36.7% in the final postintervention observation (p ¼.002). The Effect of Simulation-Based Education on Critical Care Nurses Knowledge As part of the study by Ford et al. (2010), 24 critical care nurses answered a multiple-choice quiz before and after the educational interventions. After the simulation-based educational intervention, mean quiz scores increased from 7.45 0.82 to 9.09 0.94 (p <.001). Mean quiz scores increased from 6.91 1.62 to 8.83 1.03 (p ¼.002) following a didactic lecture. Mean quiz scores were significantly improved after both the simulation-based and the lecture-based educational interventions. The difference between two types of intervention was not statistically significant (p ¼.334). Discussion The main findings were that the effectiveness of simulation-based education in critical care nurses continuing education is still uncertain and that critical care education requires new and more effective evidencebased teaching methods to ensure patient safety and the quality of care. The one included study (Ford et al., 2010) provides new and important information about the relationship between simulation-based education and clinical outcomes. The study demonstrated a clear improvement in clinical outcomes during a 2- to 3-month follow-up period: A significant decrease (p <.001) in medication administration error rates was observed, whereas the didactic lectures did not appear to lower error rates, which has important implications for patient safety and thus the quality of care. Moreover, Ford et al. (2010) showed a significant increase in medication administration error rates (p ¼.002) after the didactic lecture in the final postintervention observations, although the reason for this was unclear.

e5 The effectiveness of simulation-based education on level of knowledge compared with other educational interventions is difficult to assess because of a lack of robust evidence. In contrast to recent studies of undergraduate nursing students (Cant & Cooper, 2010; Lapkin et al., 2010), the results of Ford et al. (2010) did not indicate any significant differences (p ¼.334) in the level of critical care registered nurses knowledge after the different educational strategies. According to Blackwood et al. (2010), lack of knowledge jeopardizes patient safety and thus needs to be considered more frequently. The current body of literature is complicated by a lack of robust evidence. Few randomized controlled trials (RCTs) have been reported (Lapkin et al., 2010; Solnick & Weiss, 2007; Zigmont, Kappus, & Sudikoff, 2011), because of limitations in sample size calculation (power analysis) and sampling, recruitment and randomization of participants, as well as matching (Solnick & Weiss, 2007) and blinding. The most commonly used methods reported in the literature are direct observation and questionnaires (McGaghie et al., 2010; Solnick & Weiss, 2007) or quizzes (Ford et al., 2010). The data of Ford et al. (2010) were collected and categorized according to the United States Pharmacopeia MEDMARX system, which is widely used (Santell, Hicks, McMeekin, & Cousins, 2003). However, there is a current lack of suitable scenarios and instruments for testing the validity and reliability of results (Decker, Sportsman, Puez, & Billings, 2008; Kardong-Edgren, Adamson, & Fiztgerald, 2010; Kim, Neilipovitz, Cardinal, Chiu, & Clinch, 2006; Lapkin et al., 2010; Solnick & Weiss, 2007; Zhang et al., 2011). Conclusion The effect of simulation-based education on critical care nurses knowledge and skills is still questionable because of a lack of published studies and robust evidence, which play a major role in determining the reliability of the results and the validity of estimates of effects. According to Ford et al. (2010), simulation-based education performs well compared with didactic lectures and seems to offer significant advantages for patient safety and the quality of care by increasing critical care nurses adherence to recommendations on safe medication administration. Limitations The search strategy focused on studies written in English, Swedish, or Finnish published during the past 10 years (2002e2011), which may have led to publication or language bias (Centre for Reviews and Dissemination, 2009). However, no studies in other languages were identified. The quality of the original study was carefully assessed to ensure the validity of the review despite the lack of RCT designs. Piloting of the study selection process was not performed (Centre for Reviews and Dissemination, 2009). Implications for Research Further research is needed to design multicenter, RCT follow-up studies to evaluate the effectiveness of simulation-based education versus other educational interventions in critical care settings, a largely unexplored area even though intensive care units represent one of the most complex environments in health care facilities. Because of the invasive nature of critical care treatments and procedures, several areas have been identified as needing improvement in order to enhance safety, knowledge, and skills. These areas include catheterization, mechanical ventilation, and infection control (because critically ill patients, owing to the severity of their medical conditions, are at high risk of complications), as well as multiprofessional teamwork, situational awareness, decision making, critical thinking, and self-confidence. Moreover, a universal method of outcome measurement is required (e.g., standardized scenarios and instruments, variations in the number and duration of simulations, and standardized measurement and follow-up times) to facilitate further evaluation of the relationship between clinical outcomes and simulation-based education. Acknowledgments The authors wish to acknowledge the assistance of the library information specialist Sirpa Grekula from the Oulu University Medical Library. M.J., M.K., and H.K. contributed to the study design, data collection and analysis, and manuscript preparation. References Blackwood, B., Albarran, J. W., & Latour, J. M. (2010). Research priorities of adult intensive care nurses in 20 European countries: A Delphi study. Journal of Advanced Nursing, 67, 550-562. http://dx.doi.org/10. 1111/j.1365-2648.2010.05512.x. Cant, R. P., & Cooper, S. J. (2010). Simulation-based learning in nurse education: Systematic review. Journal of Advanced Nursing, 66(1), 3-15. PMID: 20423432. Centre for Reviews and Dissemination. (2009). Systematic reviews: CRD s guidance for undertaking reviews in health care. York, UK: York Publishing Services. Cronenwett, L., Sherwood, G., Barnsteiner, J., Disch, J., Johnson, J., & Mitchell, P. (2007). Quality and safety education for nurses. Nursing Outlook, 55, 122-131. PMID: 17524799. Decker, S., Sportsman, S., Puez, L., & Billings, L. (2008). The evolution of simulation and its contribution to competency. Journal of Continuing Education in Nursing, 39(2), 74-80. http: //dx.doi.org/10.3928/00220124-20080201-06. Ford, D. G., Seybert, A. L., Smithburger, P. L., Kubulinsky, L. R., Samosky, J. T., & Kane-Gil, S. L. (2010). Impact of simulation-based learning on medication error rates in critically ill patients. Intensive Care Medicine, 36, 1526-1531. PMID: 20300731.

e6 Gaba, D. M. (2004). The future vision of simulation in health care. Quality & Safety in Health Care, 13(Suppl 1), i2-i10. http: //dx.doi.org/10.1136/qshc.2004.009878. Jeffries, P. R. (2005). A framework for designing, implementing, and evaluating simulations used as teaching strategies in nursing. Nursing Education Perspectives, 26, 96-103. PMID: 15921126. Joanna Briggs Institute. (2008). Critical appraisal checklist for cohort/case control appraisal. In Joanna Briggs Institute reviewers manual (pp. 151). Adelaide, Australia: Author. Kardong-Edgren, S., Adamson, K. A., & Fitzgerald, C. (2010). A review of currently published evaluation instruments for human patient simulation. Clinical Simulation in Nursing, 6, e25-e35. http: //dx.doi.org/10.1016/j.ecns.2009.08.004. Kim, J., Neilipovitz, D., Cardinal, P., Chiu, M., & Clinch, J. (2006). A pilot study using high-fidelity simulation to formally evaluate performance in the resuscitation of critically ill patients: The University of Ottawa Critical Care Medicine, High-Fidelity Simulation, and Crisis Resource Management I Study. Critical Care Medicine, 34, 2167-2174. PMID: 16775567. Lapkin, S., Levett-Jones, T., Bellchambers, H., & Fernandez, R. (2010). Effectiveness of patient simulation manikins in teaching clinical reasoning skills to undergraduate nursing students: A systematic review. Clinical Simulation in Nursing, 6(6), e207-e222. http://dx.doi.org/10. 1016/j.ecns.2010.05.005. Lupien, A. E. (2007). High-fidelity patient simulation. In M. J. Bradshaw, & A. J. Lowenstein (Eds.), Innovative teaching strategies in nursing and related health professions (4th ed). Sudbury, MA: Jones and Bartlett. McGaghie, W., Issenberg, S., Petrusa, E., & Scalese, R. (2010). A critical review of simulation-based medical education research: 2003e2009. Medical Education, 44(1), 50-63. http://dx.doi.org/10.1111/j.1365-2923.2009.03547.x. McGowan, J., & Samson, M. (2005). Systematic reviews need systematic searches. Journal of Medical Library Association, 93(1), 74-80. PMID: 15685278. Mellin-Olsen, J., Staender, S., Whitaker, D. K., & Smith, A. (2010). The Helsinki Declaration on patient safety in anaesthesiology. European Journal of Anesthesiology, 27(7), 592-597. PMID: 20520556. Rall, M., van Gessel, E., & Staender, S. (2011). Education, teaching & training in patient safety. Best Practice & Research Clinical Anesthesiology, 25, 251-262. PMID: 21550549. Santell, J. P., Hicks, R. W., McMeekin, J., & Cousins, D. D. (2003). Medication errors: Experience of the United States Pharmacopeia (USP) MEDMARX reporting system. Journal of Clinical Pharmacology, 43, 760-767. PMID: 12856391. Simerka, P. (2009). Council Recommendation of 9 June 2009 on patient safety, including the prevention and control of healthcare associated infections. Luxembourg: Council of the European Union, 9-6-2009. European Commission. Resolutions, recommendations and opinions. Retrieved July 27, 2012, from http://ec.europa.eu/health/patient_safety/ docs/council_2009_en.pdf. Solnick, A., & Weiss, S. (2007). High fidelity simulation in nursing education: A review of the literature. Clinical Simulation in Nursing, 3, 41-45. http://dx.doi.org/10.1016/j.ecns.2009.05.039. Zhang, C., Thompson, S., & Miller, C. (2011). A review of simulationbased interprofessional education. Clinical Simulation in Nursing, 7(4), e117-e126. http://dx.doi.org/10.1016/j.ecns.2010.02.008. Zigmont, J. J., Kappus, L. J., & Sudikoff, S. N. (2011). Theoretical foundations of learning through simulation. Seminars in Perinatology, 35, 47-51. PMID: 21440810.