How To Test A Nurse'S Skill Level

Transcription

1 COMPARING NEW BSN RN SELF SKILLS ASSESSMENT TO ACTUAL SKILLS DEMONSTRATION JEAN ADAIR, MSN, MS, RN*, LIN HUGHES, PHD, MSN, BSN, RN, SUE DAVIS, MS, BSN, RN, AND MARY WOLCOTT-BRECI, MSN, RN* The purpose of the study was to compare the self-skills assessment with the skill competence during an actual skills demonstration of newly hired bachelor of science in nursing (BSN) registered nurse graduates. This retrospective study included 32 randomly selected BSN registered nurse graduates from January 2010 to December 31, The participants were already hired into a midwest health system. Because this was a retrospective study, no demographic data were collected, and no consent from participants was needed. This study included a clinical skills check list where the participants rated themselves on specific skills utilizing a Likert scale ranging from 1 (no knowledge) to 4(able to perform independently). The same clinical check list was utilized by an expert registered nurse when the skill was demonstrated. This study compared the difference between the subject's self-rating of skills and the clinical demonstration of the skills. We used t tests in the analysis to demonstrate the differences between the participant's self-rating of skills and the expert evaluation of the clinical demonstration of the skills. The data were inserted into the Statistical Package for the Social Sciences 19 software program to assist in the analysis process. The study demonstrated 17 significant differences in the skills ratings between the participant and competency demonstration of new BSN graduates. These significant results (2 tailed) ranged from.000 to.048.the 17 out of 46 specific skills where differences were noted included the following: staple removal, nasal pharyngeal suctioning, urinary catheter specimen collection, site care dressing change, urinary catheter irrigation, Juzo application and measurement, 5-lead telemetry, oral airway insertion, hemovac/jackson Pratt, oral pharyngeal suctioning, urinary catheter insertion, dry suction chest drainage, bed to cart/slider board, urinary catheter removal, antiembolism stockings, measurement and application, removal of iv and sit-and-stand alarm. Overall, the participants rated their skill levels lower in 15 out of 17 significant skills when compared with their competency assessment (t test: 3.284, df =31,P =.003). In two skill ratings (urinary catheter specimen collection and oral pharyngeal suction), the participants rated themselves higher than the competency demonstration. Two skills that had a mean participant and expert score between 1 (no knowledge) and 2 (able to perform with 1-to-1 coaching) were oral airway insertion and dry suction chest drainage. Some possible reasons why the participants rated *Learning Consultant, Nebraska Methodist Health System, Omaha, NE. Dean of Nursing at Nebraska Methodist College of Nursing and Allied Health, Omaha, NE. Service Leader of Learning Center and Employee Health, Nebraska Methodist Health System, Omaha, NE. No funding or conflict of interest from the authors. Address correspondence to Dr. Adair: Learning Consultant, Nebraska Methodist Health System, 8601 West Dodge Road, Suite #18, Omaha, NE [email protected] /13/$ - see front matter Journal of Professional Nursing, Vol 30, No. 2 (March/April), 2014: pp Elsevier Inc. All rights reserved.

2 SELF SKILLS ASSESSMENT TO ACTUAL SKILLS DEMONSTRATION 181 themselves lower could be the use of different or unfamiliar terms or uncertainty of the procedure at a different health institution. Some newly graduated BSN nurses may have not performed the skills on a regular basis or only in simulation. (Index words: Nursing skills; New BSN nursing graduates; Competency; Safe clinical skills; Nursing orientation; Technical skills) J Prof Nurs 30: , Elsevier Inc. All rights reserved. New Nurses' Perception of Skill Competence Compared With Expert Evaluation QUALITY AND SAFETY Education for Nurses (QSEN) emphasizes patient-centered care, teamwork and collaboration, evidence-based practice, quality improvement, safety, and informatics (Cronenwett et al., 2007). Assessing clinical skills of new bachelor of science in nursing (BSN) registered nurse (RN) graduates usually begins their individualized nursing orientation to a nursing position. This clinical assessment may benefit the individual nurse, the health system and, potentially, influence the patient care outcomes that are promoted by QSEN. Yet, nursing literature notes that the newly graduate nurse feels less prepared for today's demanding work force, entering the profession as a novice, with the first few months up to a year as a crucial time in the new nurse's professional life (Gerrish, 2000; Marshburn, Engelke, & Swanson, 2009; Scott, Engelke, & Swanson, 2008; Wangensteen, Johansson, & Nordstrom, 2008). Not only is nursing practice becoming more complex, but the employers are requiring qualified and competent staff nurses to provide the safe patient care (Tzeng & Ketefian, 2003). There are transition programs being designed to help with this bridge from education into practice, yet few have included the new graduates' perception of competence in technical skills as a form of evaluation in the program (Burns & Poster, 2008; Lofmark, Smide, & Wikblad, 2006; Marshburn et al., 2009). Greenberger, Reches, and Riba (2005) noted that nursing graduates generally perceive that they are technically competent. However, the competence in technical skills seemed to depend on the type of nursing program of the graduated nurse, the chance to practice the skill in nursing school, and opportunities for employment in a health care institution in which the skills can be utilized. Boxer and Kluge (2000) investigated the most frequently used clinical skills for first year practicing RNs and further studied which of these skills were considered essential. The researchers' findings were interesting with the fact that some of the skills that were considered most essential were not performed very often by the new nurses, which gives credence for assessment of technical skills by experts. Through this research study, the researchers wanted to add to the body of knowledge of nursing skill competence and address the perception of the new BSN RN graduate regarding their skill level compared with the actual skills demonstration, assessed by an expert nurse. Many of the technical skills assessed in this study were considered frequently used and/or essential by the participants in the study by Boxer and Kluge (2000). Purpose The purpose of the study was to compare the selfassessment of skills by new BSN RNs to an actual skill competence demonstration. The expert RN at the learning assessment center completes this skills evaluation on every nurse who is hired at the health system. Design This retrospective study included 32 randomly selected BSN RNs from January 2010 to December 31, 2010 who were hired as new graduates by a health system in the midwest. Because this was a retrospective study and no demographic data were collected, nor a consent form signed, demographic data could not be collected. The study was approved by the health system's institutional review board. Methods The 32 novice RN participants were selected using a random numbers table from an electronic listing of all newly hired BSN RN graduates during the calendar year of Each new hire was scheduled to spend time at the institution's learning assessment center before specific unit orientation at the hospital. During this time, each participate completed the clinical skills checklist, which assessed the novice nurse's own perception of their ability to perform 46 specific skills. The participants rated themselves utilizing a Likert scale ranging from 1 (no knowledge of the skill) to4(able to perform the skill independently). The same Likert-based clinical skills checklist was then utilized by an expert RN at the learning assessment center during actual skill demonstration by the novice nurse. The expert nurses maintained interrater reliability by conscientiously following each step of the 46 clinical skill procedures as stated in Perry and Potter (2010). We used t tests in the analysis to demonstrate the differences between the participant's self-rating of skills and the expert evaluation of the clinical demonstration of the skills. The data were inserted into the Statistical Package for the Social Sciences 19 software program to assist in the analysis process. Findings The study demonstrated 17 significant differences in the skills ratings between the participant self-assessment and competency demonstration of the new BSN graduates. These significant results (two tailed) ranged from

3 182 ADAIR ET AL P =.000 to.048. The 17 out of 46 specific skills where differences were noted included the following: staple removal, nasal pharyngeal suctioning, urinary catheter specimen collection, site care dressing change, urinary catheter irrigation, Juzo application and measurement, use of five-lead telemetry, oral airway insertion, management of a hemovac/jackson Pratt drain, oral pharyngeal suctioning, urinary catheter insertion, management of dry suction chest drainage, use of a bed to cart/slider board, urinary catheter removal, antiembolism stockings measurement and application, removal of an intravenous (iv) catheter, and use of a sit-and-stand alarm (see Table 1). The skills in which the participants were most comfortable included the following: gait belt application, pulse oximetry application, demonstrated use of personal protective equipment (PPE), handwashing technique, removal of a urinary catheter, intramuscular (IM) and subcutaneous (SQ) medication administration, iv catheter removal, and iv push medication administration. There were seven of the nine skills in which the participants were most confident that also reflected the expert's evaluation in the competence of the skill. There were two of the nine most confident skills (removal of urinary catheter and removal of the iv catheter) in which the experts significantly rated the participants higher in the demonstrated competence as shown in the Table 2 below and noted by an asterisk. The participants had the least confidence in the following skills: intermittent gastric gavage/bolus feeding, oral airway insertion, suctioning of a tracheostomy tube, management of a tracheostomy including a dressing change and change of a tracheostomy tube, and management of dry suction chest drainage (Table 3). These same skills reflected a lower score on the skills checklist, with two demonstrating significance. Overconfidence also may be reflected in the following skills: urinary catheter specimen collection, oral pharyngeal suctioning, and tracheostomy suctioning. In these skills, the expert rated the participant lower than the participant's self-assessment of the skill. Discussion Tzeng and Ketefian (2003) define nursing competence as personal skills developed through professional nurse training courses and is considered to be an outcome of these courses (p. 510). Yet, the literature review and meta-analysis by Watson, Stimpson, Topping, and Porock (2002) cite the continual need to define clinical competence. The authors feel that there are issues with the research studies that have assessed clinical competence, citing problems with subjectivity, bias because of personal knowledge of the nurse being assessed, judgment based on a short assessment, and emotional overtones such as testing anxiety on the part of the new nurse. Because nursing requires complex combinations of knowledge, performance, skills and attitudes, a holistic definition of competence need to be agreed upon and operationalised, according to Cowan, Norman, and Coopamah (2005, p. 355). Nursing literature suggests a multifaceted approach to the assessment of clinical competence using simulation as well as clinical and educational evaluation that is pertinent to clinical nursing practice. But Watson et al. (2002) suggest that there are questions that still remain. What should be assessed in order to define clinical competence and what is the best methodology to use to avoid the problems of socialization bias? These authors promote the use of self-assessment as one the methods to be used in any multifaceted evaluation of clinical competence (p. 424). This research study may have added to the knowledge base of clinical competence Table 1. Significant Differences Between Participant and Expert Rating Nurse Consult Significance Variable n =32 Mean SD Mean SD t test df (two tailed) Staple removal Nasal pharyngeal suctioning Urinary catheter specimen collection Site care dressing change Urinary catheter irrigation Juzo application and measurement Five-lead telmetry Oral airway insertion Hemovac/Jackson Pratt Oral pharyngeal suctioning Urinary catheter insertion Dry suction chest drainage Bed to cart slider board Urinary catheter removal Antiembolism stockings measurement and application Removal of iv Sit-and-stand alarm * Significance of P =.05 or lower.

4 SELF SKILLS ASSESSMENT TO ACTUAL SKILLS DEMONSTRATION 183 Table 2. Most Confident Skills by Participants Skill Participant mean (n = 32) Expert mean t test Significance Gait belt Pulse oximetry PPE Handwashing Removal of urinary catheter *.032 * IM injection SQ injection Removal of iv *.044 * iv Push * Significance of P.05. regarding specific skills that may define the concept. This study also promoted a multimodal approach of evaluation with the use of both self-assessment and use of an objective measure to rate the clinical skills by an expert. Clinical competent nurses are highly valued personnel in the health system. The American Nurses Association (ANA) has defined some of the nurse-sensitive quality indicators as the following: nursing hours per patient day; RN, licensed practical nurse (LPN), and unlicensed assistive personnel (UAP) hours per patient day; nursing turnover; nosocomial infections (hospital and community acquired); patient falls (with injury and without injury); pressure ulcer rate (hospital and community acquired); RN education/certification; RN survey of job satisfaction scales and practice environment scales; use of restraints; and the staff mix of RN, LPN, and UAP and percentage of agency staff (ANA, 2000). These ANA nurse-sensitive quality indicators are deeply imbedded with the need for technical skill competence to maintain safe and effective patient care outcomes. This research study may have helped define what to assess in the realm of clinical competence. Overall, the participants in this study rated their skill levels lower in 15 out of 17 skills when compared with the expert competency assessment (t test = 3.284, df = 31, P =.003). This is further validation that new nursing graduates may lack the confidence in themselves initially. Heslop, McIntyre, and Ives (2001) state that when RNs invest much time and energy into the performance of technical procedures, then there is little energy left for the other priority nursing functions such as clinical judgment. In two of the significantly different skill ratings (urinary catheter specimen collection and oral pharyngeal suctioning), the participants rated themselves higher than the expert competency demonstration. Overconfidence may be a concern with these skills and the tracheostomy suctioning skill in these new BSN graduate nurses. Educators in the initial program and unit-specific preceptors would want to be aware of this discrepancy and review these skills more closely in the clinical practicum and in the skills laboratory. Five skills that had a mean participant and expert score between 1 (no knowledge) and 2 (able to perform with one to one coaching) were intermittent gastric gavage/bolus feeding, oral airway insertion, tracheostomy suctioning, tracheostomy management, and dry suction chest drainage. Some possible reasons why the participants rated themselves lower could be the use of different or unfamiliar terms or uncertainty of the procedure at a different health institution. Some newly graduated BSN nurses may have not performed the skills on a regular basis or only in simulation. Again, both education and practice should focus on these skills. Greenberger et al. (2005) suggested that it may be effective to schedule clinical rotations at an institution for longer periods as a student nurse in order to gain confidence and competence in technical skills rather than taking more energy to orient to a new facility. These authors also emphasized the importance of clinical faculty acting as expert role models. Nine skills were perceived and demonstrated at a level between 3 (able to perform with minimal prompting) and 4 (able to perform independently). It could be concluded that educators are successfully teaching and reinforcing the skills of gait belt use, pulse oximetry, personal protection equipment, handwashing, removal of urinary catheters, IM injection, SQ injection, removal of an iv catheter and iv push of medications. By identifying the successful transfer of these skills from education into practice, preceptor programs can be reassured that less time needs to be devoted to reteaching these skills and Table 3. Least Confident Skills by Participants Skill Participant mean (n = 32) Expert mean t test Significance Intermittent gastric gavage/bolus feeding Oral airway insertion * Tracheostomy suctioning Tracheostomy management Dry suction chest drainage management * * Significance of P.05.

5 184 ADAIR ET AL more emphasis can be placed on increasing the confidence and competence of the other identified skills. This in itself could focus the unit orientation and have significant financial benefit. Conclusion The definition of clinical competence is still in its infancy and continues to evolve as health care continues to change. Competencies, which might change along with environmental changes, comprise a group of broad abilities and practical skills (Tzeng & Ketefian, 2003, p. 510). The highest ranking competency in the study by Tzeng and Ketefian was general professional technical skills. Nursing competencies also seem to vary according to the nursing units, type of services provided by the hospital, or the health care system (Meretoja, Leino-Kilpi, & Kaira, 2004). In the United States, with such specialized hospital units, these technical skills become more specifically defined and complex. With such heavy demands on the newly graduated nurse, technical competence is essential. It is imperative that employers understand that the new nurse's perception of skill ability is as important to understand as the actual competence of the technical skill itself. Both are important to assess in the new graduate. Overconfidence could be costly to patient outcomes with the new nurse being unaware of safety measures or precautions to be taken in the performance of specific procedures. Lack of confidence in skill performance could take its toll as well in relationship to the nurse's energy, use of resources, and time needed to perform nursing interventions. This study was based on a small representation of newly graduated BSN nurses and cannot be generalized to the general population. The authors also declare that the objective tool, the clinical skills checklist, used by the expert nurses in this study does not completely rule out the possibility of socialization bias. Future research should be devoted to a larger sample size and researching the possible demographic connections that may influence the perception and actual competence of technical nursing skills. Other areas that should be assessed in new nursing graduate nurses would be interdisciplinary communication, self-coping skills, team building, health system knowledge, and the use of clinical judgment. These are crucial skills needed for this age of informatics and dramatic health care changes. References American Nurses Association. (2000). Nursing quality indicators beyond acute care: Literature review. Washington, DC: American Nurses Publishing. Boxer, E., & Kluge, B. (2000). Essential clinical skills for beginning registered nurses. Nurse Education Today, 20, Burns, P., & Poster, E. (2008). Competency development in new registered nurse graduates: Closing the gap between education and practice. The Journal of Continuing Education in Nursing, 39, Cowan, D., Norman, I., & Coopamah, V. (2005). Competence in nursing practice: A controversial concept-a focused review of literature. Nurse Education Today, 25, Cronenwett, L., Sherwood, G., Barnsteiner, J., Disch, J., Johnson, J., Mitchell, P., Sullivan, D. T., & Warren, J. (2007). Nursing outlook, 55, Gerrish, K. (2000). Still fumbling along? A comparative study of the newly qualified nurse's perception of the transition from student to qualified nurse. Journal of Advanced Nursing, 32, Greenberger, H., Reches, H., & Riba, S. (2005). Do new graduates of registered nursing programs in Israel perceive themselves as technically competent? Journal of Continuing Education in Nursing, 36, Heslop, L., McIntyre, M., & Ives, G. (2001). Undergraduate student nurses' expectations and their self-reported preparedness for the graduate year role. Journal of Advanced Nursing, 36, Lofmark, A., Smide, B., & Wikblad, K. (2006). Competence of newly-graduated nurses A comparison of the perceptions of qualified nurses and students. Journal of Clinical Nursing, Marshburn, D., Engelke, M., & Swanson, M. (2009). Relationships of new nurses' perceptions and measured performance-based clinical competence. The Journal of Continuing Education in Nursing, 40, Meretoja, R., Leino-Kilpi, H., & Kaira, A. (2004). Comparison of nurse competence in different hospital work environments. Journal of Nursing Management, 12, Perry, A., & Potter, P. (2010). Clinical nursing skills & techniques. (7th Ed.). St. Louis, MO: Mosby Elsevier. Scott, E., Engelke, M., & Swanson, M. (2008). New graduate nurse transitioning: Necessary or nice? Applied Nursing research, 21, Tzeng, H., & Ketefian, S. (2003). Demand for nursing competencies: An exploratory study in Taiwan's hospital system. Journal of Clinical Nursing, 12, Wangensteen, S., Johansson, I., & Nordstrom, G. (2008). The first year as a graduate nurse An experience of growth and development. Journal of Clinical Nursing, Watson, R., Stimpson, A., Topping, A., & Porock, D. (2002). Clinical competence assessment in nursing: A systematic review of the literature. Journal of Advance Nursing, 39,