Electronic Standardize Testing for Advanced Practice Registered Nursing (APRN) Programs Pamela Willson, PhD, RN, FNP, BC Prairie View A&M University, Associate Professor Elsevier, Assessment HESI, Director of Research
Objectives Summarize a standardized APRN end of program examination s predictive validity. Critique potential contextual implementation strategies for including standardized examinations within APRN.
Background There are more than 158,000 practicing NPs. Close to 9,203 new NPs are prepared each year at over 325 colleges and universities in the U.S. To assure a match between education and clinical competencies, certification is a commonly legislated practice eligibility mandate.
Graduate nursing faculty are implementing standardize certification predictive testing, exit exam benchmarking, and remediation strategies to facilitate student success on national certification exams. A major indicator of nursing programs effectiveness is the pass rate on national certification examinations among first-time candidates.
Twenty years of evidence supports use of standardized examinations for undergraduate end of program assessment to facilitate knowledge development, ensure competence of their new graduates, and demonstrate organizational and curriculum effectiveness. Standardize examination use in graduate education lags behind that of undergraduate education, providing limited evidence to guide faculty.
Purpose The aim of this study was to establish the predictive validity of a standardized examination for APRN certification success in a national United States sample. To determine implementation strategies used by their schools.
Evidence-based Testing Products Should contain: Reliable and valid test items Matched certification body test blueprints Established national benchmark or cutscore
Reliability Reliability Estimates dynamically measured Calculated for each HESI exam administered Item Difficulty Item Discrimination Point Biserial Correlation Coefficient Kuder-Richardson 20 (KR-20) Morrison, S., Adamson, C., Nibert, A., & Hsia, S. (2004) HESI Exams: An overview of reliability and validity. CIN: Computers, Informatics, Nursing. (22)4, 220-226.
Exam Blueprints HESI APRN Exam Blueprints HESI Standard Scoring Categories NONPF Competencies ANCC & AANP Exam Content Outlines National Organization of Nurse Executives Exams Developed Family Nurse Practitioner Adult Nurse Practitioner Acute Care Nurse Practitioner* Nursing Executive* 3Ps Patho/Pharm/Physical Diagnosis* * Piloting Exam
HESI APRN Exam Reliabilities and Average Item Uses Family NP Exams KR-20 ~ 0.932 Item uses = ranges from 466 to 2394 HESI Recommended Benchmarks Recommended Score - 800 and above Minimally Acceptable Score - 750
Methods Electronic surveys were mailed to Directors/Chairs of 35 graduate nursing schools that administered Elsevier HESI Advanced Practice Registered Nurse (APRN) Exam for Family and/or Adult APRN students during the 2008 2009 academic year. Data were collected regarding students APRN national certification examination outcomes and the schools implementation strategies and policies.
Methods After obtaining Institutional Review Board approval, a cohort of students was invited to participate in testing and report review. The Elsevier HESI Family Nurse Practitioner Exam (FNP) consisting of 110 items was taken electronically and individualized score reports were printed at the conclusion of the exam. Faculty reviewed exam outcomes and reports with the participants and action plans were developed.
Instruments Elsevier HESI APRN FNP 100-item APRN Comprehensive Exams 10 pilot items Matched to AANP and ANCC test blueprints Identify specific areas for remediation Provide an objective measure of program outcome achievement Contribute to overall curriculum evaluation
Instruments Validity Study Questionnaire 14-item Policy Benchmarks Consequences Remediation Use of Score reports Satisfaction with tool Student & Program
Survey 13-item survey Implementation strategies Count for grade/percentage Timing Score report Program policy Remediation Sent out to 35 programs 9 programs 141 students
National Survey Results Program types 3 FNP 3 ANP 3 FNP & ANP Three schools count exam for 10% of grade One school had an Exit Testing Policy Benchmark was set at 800 Consequences
Results Certification outcomes were reported for 141 APRN students 96 Family APRN students 45 Adult APRN students APRN HESI scores Ranged from 484 to 1039 Mode of 805 Mean of 796 Standard deviation 111.4
Summary Scoring Report
Study Results Elsevier HESI APRN FNP Exam Results Descriptive Statistics N Minimum Maximum Mean Std. Deviation 141 484 1039 796.09 113.390
Exam Scoring Categories Number & Percent of Students by Scoring Category (N = 141) Scoring Category HESI Score N Percent Category A/B 900->1000+ 22 16% Category C 850-899 23 16% Category D 800-849 30 21% Category E/F 700-799 39 27% Category G/H < 699 28 20%
HESI Scores & Certification Number & Percent of Students by Scoring Category (N = 141) Scoring Category HESI Score N Predictive Rate Category A/B 900->1000+ 22 100% Category C 850-899 23 100% Category D 800-849 30 100% Category E/F 700-799 38 100% Category G/H < 699 28 56%
Identify Learning Needs
Competency Outcomes Standardize APRN predictive exams provided valuable student competency data that allow faculty to address student knowledge gaps with remediation and clinical experiences.
Course & Program Evaluation In addition, standardize testing provides outcome measures for curriculum evaluation, program accreditation, and program policy development.
Program Evaluation Use to: Document the evidence for decision making Track and Trend outcomes over several cohorts of same and/or different specialty tracks Adds richness of details at the program and course level for quality assurance and accreditation bodies
Summary A national multi-site United States sample of graduate programs has found that electronic standardized end of program assessments are highly accurate at predicting APRN certification success. These findings are consistent with preliminary benchmark setting studies for APRN HESI minimum scores of 750 or higher. Faculty s implementation strategies, policy development, and methods for curricular evaluation were discussed.