2012 Scientific Evidence for Elsevier HESI Exams Nurse educators use evidence-based teaching strategies to deliver the core nursing curriculum content. Elsevier HESI provides Admission Assessment (A 2 ), Exit (E 2 ), Specialty, and Custom examinations that are evidence-based. This allows faculty to consistently and authoritatively evaluate their student s learning, give direction for remediation, and evaluate the strength of their curriculum. The assessment and evaluation processes are only accurate if valid and reliable exams are used. Investigators have found Elsevier HESI examinations, including the Admission Assessment, Exit Exam, Specialty, and Custom exams, to be successful measures for benchmarking program outcomes, measuring student achievement, guiding remediation prior to licensure candidacy, and facilitating effective hospital orientation. Elsevier HESI Tests Are Valid and Reliable Examinations Providing valid and reliable test items for each HESI exam is an important, methodical process that guarantees the exams are reflective of the ability to deliver the highest quality of critical thinking test items. Student re-testers receive reliable equivalent test versions (RN and PN) with items that they have not previously seen. Reliability was determined by conducting item analyses on each exam and statistically calculating reliability. Elsevier end-of-program testing and Exit Exams range in the highest categories for estimated reliability coefficients using the Kuder Richardson Formula 20 (KR-20) and range from 0.90 to 0.94. Elsevier HESI Testing Improves Nurses National Licensure Pass Rates Studies have determined the accuracy of Elsevier HESI exams. All 9 HESI RN Exit Exams validity studies were found to be consistently accurate at predicting NCLEX success among the students scoring 900 and above. Table 1 outlines RN studies conducted, sampling methods, number of participants, and study outcomes. The RN predictive accuracy ranges from 96% to 99%. Table 2 outlines the PN studies, also highly accurate at predicting NCLEX-PN success. Table 1: Predictive Accuracy of Registered Nurse (RN) HESI Exit Exam Validity Studies RN Study Sampling Participants Predictive Accuracy Methodology One convenience 2,555 97.3 Two convenience 3,296 96.5 Three convenience 5,588 97.6 Four convenience 5,903 98.3 Five convenience 9,695 97.8 Six convenience 10,147 96.4 Seven stratified random* 4,383 99.2 Eight stratified random* 4,134 98.3 Nine stratified random* 3,790 96.6 * Stratified by program type i.e., Baccalaureate, Associate Degree, and Diploma 2012 Scientific Evidence 1
Table 2: Predictive Accuracy of Practical Nurse (PN) HESI Exit Exam Validity Studies PN Study Sampling Participants Predictive Accuracy Methodology One convenience 170 100.0 Two convenience 456 97.9 Three convenience 689 99.1 Four convenience 897 99.4 Five random 1,149 99.5 Six random 962 98.4 Three investigations found that Advanced Practice Registered Nurses (APRN) students scoring 800 and higher had a 100% certification pass rate regardless of certifying body - American Academy of Nurse Practitioners (AANP) or American Nurses Credentialing Center (ANCC) certification exams. Table 3 outlines APRN studies conducted, sampling methods, number of participants, and study outcomes. Table 3: Predictive Accuracy of APRN End of Program HESI Exam Validity Studies APRN Study Sampling Participants Predictive Accuracy Methodology Family Nurse multi-site, convenience 49 100.0 Practitioner Family & Adult NP convenience 170 100.0 Family & Adult NP national convenience 120 100.0 Implementation Science The Implementation of HESI Exams includes student remediation strategies, policy setting, and curriculum evaluation that is based on the evidence. Specialty Exams Improve NCLEX-RN Pass Rates Results: Sixty-three participating schools, representing a total of 3,790 students, who took the HESI Exit Exam and the NCLEX-RN licensure exam during 2008-2009 were analyzed by students who also took one or more HESI Specialty Exams during their course work. The mean Exit Exam score for students who used Specialty Exams during their programs was 865.7, and the mean Exit Exam score for non-users was 837.3. Using the benchmark score of 850 and above, findings indicated that all eight Specialty Exams were significantly predictive (p.0001 to.0034) of NCLEX-RN success. The top three most predictive specialty exams were Critical Care (OR= 11.16), Pediatrics (OR= 5.51), and Medical-Surgical (OR= 5.50). Schools of nursing use Specialty Exams as a course test, course final exam, remediation, and for curriculum evaluation. Conclusions: HESI Specialty Exams were found to predicting NCLEX-RN success. Significantly more students who took HESI Specialty Exams passed the NCLEX-RN on their first attempt than those who did not take specialty exams. (1) 2012 Scientific Evidence 2
Repeat Testing After Remediation Results: Students in testing groups taking only one exam (version 1, n=26) had a 92% NCLEX-RN pass rate. Students pass rates for versions 2 and 3 were 87% and 82%, respectively. Remediation strategies most often implemented by nursing programs were: 1) online exam remediation, 2) remedial preparation course, 3) computer-based tutoring, 4) NCLEX-RN Review Manual, 5) NCLEX-RN preparation book, 6) tutoring, 7) re-test with a different vendor, 8) use of online case studies, and 9) repeat the course. Various strategies were implemented. Students who remediated had higher NCLEX-RN pass rates (86%) versus non-remediating students (77%). Conclusions: At risk students for NCLEX-RN failure can increase their parallel predictive exam scores with remediation. Strategies often implemented were: 1) online exam remediation, 2) remedial course, and 3) computerbased tutoring. (2) HESI Mid-Curricular Exams for Predicting Nursing Student Academic Success Results: The HESI Mid-Curricular Exams were predictive for second year nursing students (N=86) success at completing four sequential semesters of the nursing program at two campuses, NCLEX-RN success (95.2% to 95.8%), and a 15-week medical-surgical nursing capstone course. The schools programmatic success for all courses using HESI Mid-Curricular Exam scores of 800 and 900 was 80% and 81% respectively. Conclusion: The HESI Mid-Curricular Exam was useful in identifying students at risk for not completing the nursing program, HESI Exit Exam, and NCLEX-RN. The HESI Mid-Curricular benchmark of 800 yielded the best relationship of academic to NCLEX success. (3) Informed Nursing Student Remediation, Policy, and Curriculum Evaluation Results: The required passing score of 85 (equivalent to 850) for the examination was the initial benchmark set in 2003. Using aggregated data for continuous program evaluation, the passing score was raised to 900 in 2008. The progression policy has had a positive impact on improving student performance. Conclusion: Evidence-based decision-making dramatically improved NCLEX pass rates in this program. (4) Enriched Advanced Practice Nursing Students Clinical Experiences Results: The predictive validity of the HESI scores for students achieving scores >800 on the exams with their ANCC or AANP certification exam outcomes was again evaluated for accuracy, and the data analysis indicated acceptable and recommended scoring levels. Conclusion: The standardized FNP and ANP exams were effective in assessing students preparedness for the specialty accreditation exams and also provided evidenced-based measures of curricular outcomes. (5) Focused Nurse Orientations with Identified Remediation and Mentoring Topics Results: The RNs critical thinking ability significantly correlated (-.325; p=.004) with length of time in orientation, indicating that those with a higher critical thinking score required fewer clinical orientation days to meet orientation objectives. RN orientation time was decreased by 50%. Conclusions: Standardized testing assists nurse educators in meeting RN orientation needs and meeting facility goals. (6) 2012 Scientific Evidence 3
Custom Exams Evaluates Teaching Strategies of Simulation vs. Case Studies Results: The effects of simulation activities on critical thinking and self-confidence in an electrocardiogram (ECG) nursing course were measured in 140 students. Seventy students received weekly simulation in addition to lecture (350 didactic and 150 simulation minutes) and the control group (n=70) received weekly lecture (400 minutes), but no significant difference was found in overall scores or the subscales of assessment, critical thinking or therapeutic nursing interventions. Conclusion: The authors suggest that didactic instruction with static ECG practice may be a more effective than simulation to teach beginning students the ECG related critical thinking skills. (7) Admission Assessment (A 2 ) Predict Academic Success Results: Admission Assessment Exams with a benchmark of 70% were effective in determining success in first semester nursing courses. National mean scores were: 1) Math - 75, 2) Grammar - 67, 3) Reading Comprehension - 76, 4) Vocabulary - 59, and 5) composite score of 69. The language components had the highest correlation to academic success and retention rates. Conclusions: The higher the Admission Assessment Exam component and composite scores the more likely the students were to be successful in the two nursing courses. (8) HESI Validity Studies Determine National Educational Policies and Practices (9,10) Benchmark Scores on HESI Exit Exams HESI RN Exit Exam Most common benchmark was 850 or higher by 48% (32/66 schools). A benchmark of 900+ 98.3%; 850+ 97.5%; and 800+ 96.7% pass rates. HESI PN Exit Exam Most common benchmark was 850 or higher by 38.5% (10/26 schools). A benchmark of 900+ 98.4%; 850+ 98.2%; and 800+ 96.1% pass rates. Consequences of HESI Exit Exam Outcomes HESI RN Exit Exam Most frequent consequence of not meeting the benchmark was retaking the Exit Exam (n=42), failing the course/capstone (n=13), delay/deny graduation (n=21), or NCLEX candidacy (n=18). HESI PN Exit Exam Most frequently no consequence was imposed for not meeting the benchmark, followed by re-taking the exit exam (n=7), delay/deny graduation (n=4), or NCLEX candidacy (n=5). Repeat testing Repeat Exit Exams Twenty-four (36.4%) of schools did not repeat testing; 9 schools each permitted 1-2 re-takes (versions 1, 2, and 3); 24 schools allowed 4-8 re-takes Most schools (71%) required remediation, most often allowing 2-6 weeks between re-testing 2012 Scientific Evidence 4
Remediation Requirements RN Multiple strategies were used, most frequently Evolve online and computer-based tutoring (n=38), NCLEX preparation book (n=12), and case studies (n=11). PN Multiple strategies were implemented - both faculty developed and use of HESI products including Evolve online exam remediation (n=6 schools), live review (n=2), case studies (n=3), and NCLEX preparation books (n=3). References: 1. Zweighaft, (2011). Adapted from: Sigma Theta Tau International s 41st Biennial Convention. Grapevine, TX. National standardize specialty exams improve predictive Exit Exam scores and NCLEX-RN success. The presentation slides may be viewed at: http://www.nursingsociety.org/sttievents/biennialconvention/pages/41_2011_presentations.aspx Click on Dr. Elizabeth Zweighaft s presentation 2. Allen, P.G. (2010). Post-predictive exam failure remediation and NCLEX-RN success of baccalaureate nursing graduates. Paper Presentation at the 21st International Nursing Research Congress, July 12-16, 2010. Orlando, FL. 3. Harding, M., Rateau, M., & Heise, J. L. (2011). Efficacy of a Midcurricular examination for predicting nursing student academic success. CIN: computers, Informatics, Nursing. Preprint release. DOI: 10.1097/NCN.0b013e3182066458 4-5. Adapted from: Sigma Theta Tau International s 20th International Nursing Research Congress in Vancouver, Canada. (2009). Using Secure Computerized Testing to Direct Success with National Licensure, Advanced Practice Certification, and Hospital Orientation Competencies of Registered Nurses. Retrieved from: http://www.nursingsociety.org/sttievents/researchcongress/pages/pastcongresspresentations.aspx Click on the first Willson, Pamela presentation. 4. Sewell, J., Culpa-Bondal, F. & Colvin, M. (2009). Peer reviewed podium presentation. 5. Binder, B. (2009). Peer reviewed podium presentation. 6. Ryan, C., & Tatum, K. (2012). Objective Measurement of critical-thinking ability in registered nurse applicants. The Journal of Nursing Administration, 42(2), 89-94. 7. Brown, D. & Chronister, C. (2009). The effect of simulation learning on critical thinking and self-confidence when incorporated into an electrocardiogram nursing course. Clinical Simulation in Nursing, 5(1),e45-e52. Doi: 10.1016/j. ecns.2008.11.001 8. Knauss, P. & Willson, P. (2011). Mosby s Faculty Development Institute. Reliable instruments for measuring outcomes: Admission Assessment how well is it working? Podium presentation. Orlando, FL. 9. Langford, R. & Young, A. (2010). Adapted from: Sigma Theta Tau International s 21st International Nursing Research Congress. Orlando, FL. Predicting NCLEX Success with the Elsevier Evolve Exit Exam: the Eighth Study of Validity. Retrieved from: http://www.nursingsociety.org/sttievents/researchcongress/pages/2011_2010presentations.aspx 10. Langford, R. & Young, A. (2011). Mosby s Nursing Faculty Institute. Providing Valid and Reliable RN and PN Exit Exams: The 8th Validity Study. Podium presentation. Orlando, FL. 2011 Scientific Evidence 5
Resources: Ask your Elsevier Education Solutions Consultant for more information including: HESI Assessment RN brochure and HESI Assessment PN brochure HESI Assessment Testing & Remediation Research Bibliography State of the Science: HESI Remediation and NCLEX Preparation Bibliography Ninth Validity RN Exit Exam Study Summary Report Eighth Validity RN Exit Exam Study Summary Report Sixth Validity PN Exit Exam Study Summary Report Future Research: There are multiple research studies currently underway where Nurse Educators and Health Professionals are investigating testing and remediation and measuring student outcomes of HESI product use. For more details, please contact Dr. Pamela Willson, Director of Research, HESI Review and Testing at 713-346-6916 or pa.willson@elsevier.com. 2012 Scientific Evidence 6 ELS-xxx (05/29/2012)