NBCRNA FY 2012 Summary of NCE/SEE Performance and Transcript Data

Transcription

1

2 TABLE OF CONTENTS 1. INTRODUCTION TEST TAKER PERFORMANCE ON THE NCE DEMOGRAPHIC CHARACTERISTICS OF FY 2012 NCE TEST TAKER POPULATION DESCRIPTIVE INFORMATION ON NUMBER OF ANESTHESIA CASES NCE EXIT SURVEY RESULTS DEMOGRAPHIC CHARACTERISTICS OF FY 2012 SEE TEST TAKER POPULATION APPENDIX... 23

3 1. Introduction The National Board of Certification and Recertification for Nurse Anesthetists (NBCRNA) implemented computerized testing for its examinations beginning in April The NBCRNA administers two adaptive tests: the National Certification Examination (NCE) and the Self Evaluation Examination (SEE). This report presents a summary of information on students performance on the NCE and the SEE in FY 2012 (September 1, 2011 August 31, 2012). For the NCE performance (Sections 2 and 3), pass/fail outcomes are summarized by gender, age, clinical background, and type of graduate degree. Section 4 contains descriptive statistics (mean, standard deviation, etc.) summarizing the number of cases performed in various clinical areas by FY 2012 nurse anesthesia educational program graduates. Section 5 of the report provides an analysis of test takers responses on a satisfaction survey administered at the end of the NCE. The survey asks questions pertaining to test takers opinions about their registration and test experience. Section 6 contains summaries of SEE scaled scores broken down by gender, age, clinical background, type of graduate degree, and year in program. Please note that, while previous reports presented the data summarized over a calendar year (January 1 December 31), the present report presents data summarized over a fiscal year (September 1 August 31). 1

4 2. Test Taker Performance on the NCE Table 1 summarizes NCE performance for FY 2012 examinees. Table 1 shows pass rates for FY 2012 examinees, divided by first-time test takers and repeat test takers. Note that the pass rate depreciates for repeat examinees. First-time examinee pass rates for the NCE can be found in Table A1 in the Appendix of this report. Summaries of NCE total scores and subscores for FY st time test takers can also be found in Table A2 of the Appendix. Numbers of graduates and certified nurse anesthetists since 2006 can be found in Figure A1 of the Appendix. TABLE 1 PASS/FAIL SUMMARY FOR FY 2012 NCE TEST TAKERS 1 ST TIME TAKERS FREQUENCY PERCENT PASS % FAIL % TOTAL % REPEAT TAKERS PASS % FAIL % TOTAL % 2

5 Table 2 shows the distribution of test length and pass/fail status. Only 1 st time examinees are included in Table 2. The majority of test takers (66.5%) passed the test in 70 items (not including the 30 unscored pretest items). Only 3.4 percent of NCE test takers failed the test at 70 items. Approximately 13.6 percent of the test takers took the maximum test length of 140 items. TABLE 2 PASS/FAIL SUMMARY BY TEST LENGTH FOR FY ST TIME NCE TEST TAKERS FREQUENCY PERCENT PASS IN 70 ITEMS % PASS IN 71 TO 139 ITEMS % PASS IN 140 ITEMS % FAIL IN 70 ITEMS % FAIL IN 71 TO 139 ITEMS % FAIL IN 140 ITEMS % TOTAL % 3

6 3. Demographic Characteristics of FY 2012 NCE Test Taker Population Tables 3 through 6 summarize FY 2012 pass rates on the NCE by gender, age, clinical background, and degree earned. Only 1 st time examinees are included in these tables. Table 3 illustrates that approximately 63.5% of the NCE test takers in FY 2012 were female, 36.5% male. The pass rates between males and females were essentially equal. Table 4 displays the pass rate by age group and demonstrates that the pass rate decreased as examinee age increased. The average age of the FY 2012 NCE examinees was just under 33.4 years old. TABLE 3 GENDER OF FY st TIME NCE TEST TAKERS GENDER PASS FAIL TOTAL N PERCENT N PERCENT N PERCENT FEMALE % % % MALE % % % TOTAL % % % TABLE 4 AGE OF FY st TIME NCE TEST TAKERS AGE PASS FAIL TOTAL N PERCENT N PERCENT N PERCENT UNDER % % % % % % % % % 40 OR MORE % % % TOTAL % % % 4

7 Table 5 displays pass rates by clinical background. TABLE 5 CLINICAL BACKGROUND OF FY st TIME NCE TEST TAKERS CLINICAL BACKGROUND PASS FAIL TOTAL N PERCENT N PERCENT N PERCENT CCU % % % ER 0 0.0% % % ICU/CCU % % % MICU % % % NEURO ICU % % % NICU % % % OR % % % PACU % % % PICU % 1 1.3% % SICU % % % Trauma ICU % % % Other % 0 0.0% % TOTAL % % % 5

8 Table 6 displays pass rates by degree attained. Over half (52.2%) of the FY st time NCE examinees attained the MSN degree. Pass rate comparisons between different clinical settings (Table 5) and degree (Table 6) should be made with caution because some demographic subgroups feature small sample sizes. TABLE 6 TYPES OF GRADUATE DEGREE OF FY st TIME NCE TEST TAKERS DEGREE UPON COMPLETION PASS FAIL TOTAL N PERCENT N PERCENT N PERCENT MS Nursing Major % % % MS Nurse Anesthesia % % % Other Masters % % % Doctoral % % % Post Masters Certificate % % % Total % % % 6

9 4. Descriptive Information on Number of Anesthesia Cases The following tables summarize the data collected about the number of anesthesia cases performed in clinical areas, as reported on the official NBCRNA Transcript of Student Record forms submitted by program directors for their students completing nurse anesthesia educational programs FY Tables 7-17 summarize the student frequency distributions of anesthesia cases for each specific area. The organization of the tables reflects the headings and subheadings in the record of clinical experience section of the official NBCRNA Transcript of Student Record(see the Figure A2, Record of Clinical Experience in the Appendix). The same columns are replicated in Tables 7-17: The first column contains the clinical area in which cases were performed. The Number (N) column represents the number of transcripts submitted in FY The Required Number of Cases column indicates the minimum number of cases that must be completed by any applicant in order to be deemed eligible to take the NCE. If, within a given clinical area, cases were not required, but rather preferred, the suggested number of cases is indicated in brackets ([ ]). If a minimum number of cases is neither required nor preferred, a 0 is entered in this column. Please refer to the Appendix for a copy of the Record of Clinical Experience, which documents the required number of cases in each clinical area. The Minimum (Min) column indicates the smallest number of cases reported on the FY 2012 transcripts. The Mean column indicates the average number of cases reported on the FY 2012 transcripts. The Median column indicates the median number of cases (50 th percentile) reported on FY 2012 transcripts. Half of the FY 2012 transcripts contained values higher than this number, and half contained a value below this number. The SD column indicates the standard deviation in the number of cases reported on the FY 2012 transcripts. TABLE 7 SECTIONS I, II & III: TOTAL NUMBER OF CASES, HOURS OF ANESTHESIA, CLINICAL HOURS Area N Required Number of Cases Mean Standard Deviation Median Minimum Total Number of Cases Total Hours of Anesthesia Total Clinical Hours TABLE 8 SECTION IV: PATIENT PHYSICAL STATUS Area N Required Number of Cases Mean Standard Deviation Median Minimum Class I Class II Class III & IV Class V 2611 [5] Note: Number of cases enclosed in brackets ([ ]) indicate preferred number of hours, rather than required. 7

10 TABLE 9 SECTION V: SPECIAL CASES Area N Required Number of Cases Mean Standard Deviation Median Minimum Geriatric, 65+ years Pediatric, 2-12 years Pediatric, under 2 years Neonatal, under 4 weeks 2611 [5] Trauma/Emergency Ambulatory/Outpatient Obstetrical Mgmt Obstetrics, Cesarean Section Obstetrics, Analgesia for Labor Note: Number of cases enclosed in brackets ([ ]) indicate preferred number of hours, rather than required. Area N TABLE 10 SECTION VI: POSITION CATEGORIES Required Number of Cases Mean Standard Deviation Median Minimum Prone Lithotomy Lateral Sitting

11 TABLE 11 SECTION VII: ANATOMICAL CATEGORIES Area N Required Number of Cases Mean Standard Deviation Median Minimum Intra-abdominal Extrathoracic Extremities Perineal Head, Extracranial Head, Intracranial Head, Oropharyngeal Intrathoracic Heart Lung Intrathoracic, Other Neck Neuroskeletal Vascular Other TABLE 12 SECTION VIII: PHARMACOLOGICAL AGENTS Area N Required Number of Cases Mean Standard Deviation Median Minimum Inhalation Agents Intravenous Induction Agents Intravenous Agents, Muscle Relaxants Intravenous Agents, Opioids

12 Area General Anesthesia Intravenous Induction Inhalation Induction Mask Management LMA Tracheal Intubation/Oral Tracheal Intubation/Nasal Total Intravenous Anesthesia Emergence from Anesthesia Monitored Anesthesia Care N TABLE 13 SECTION IX: METHODS OF ANESTHESIA Required Number of Cases Mean Standard Deviation Median Minimum [10] Regional/Management Administration Spinal 2611 [50] Epidural 2611 [50] Peripheral 2611 [40] Experience required in each category. 10

13 TABLE 14 SECTION X: ARTERIAL TECHNIQUE Area Arterial Puncture/Catheter Insertion N Required Number of Cases Mean Standard Deviation Median Minimum Intra-arterial Blood Pressure Monitoring TABLE 15 SECTION XI: CENTRAL VENOUS PRESSURE CATHETER Area Placement Monitoring N Required Number of Cases Mean Standard Deviation Median Minimum TABLE 16 SECTION XII: PULMONARY ARTERY CATHETER Area Placement Monitoring N Required Number of Cases Mean Standard Deviation Median Minimum 2611 [5] [10] Note: Number of cases enclosed in brackets ([ ]) indicate preferred number of hours, rather than required. 11

14 TABLE 17 SECTION XIII: OTHER Area N Required Number of Cases Mean Standard Deviation Median Minimum Intravenous Catheter Placement Mechanical Ventilation Pain Management [10] Alternative Airway Management/Total Fiberoptic Total Alternative Airway Mgmt. Techniques: Other Note: Number of cases enclosed in brackets ([ ]) indicate preferred number of hours, rather than required. 12

15 5. NCE Exit Survey Results As part of NBCRNA s ongoing evaluation of the testing process, NCE test takers are asked to complete a post-examination survey regarding their test experience. The post-examination survey consists of the following: Seven statements related to pre-examination activities such as registration and scheduling, locating the testing center and interaction with test center staff. Nine statements related to examination experience such as readability, fairness of test questions, and use of testing software. Four statements related to examinee perception of the Alternative Question Formats. Three statements related to exam preparation. The majority of the survey questions involved a Likert-type rating scale which requires the respondents to indicate their level of agreement with the statement. The survey questions and format were developed by NBCRNA and representatives from its testing agency, Pearson VUE, Inc. Completion of the survey is not required as part of the examination process and is not part of the three-hour time limit. Respondents do not always answer all of the questions on the survey, as reflected by the unequal sample size across questions. Responses to the survey questions were analyzed based on a sample of NCE test takers who were administered the exit survey during the period of September 1, 2011 through July 23, 2012 Following each test administration, the test taker could also contact the NBCRNA office to address any problems or concerns related to the NCE. The first seven statements pertain to pre-examination scheduling and registration activities. About 83.3% of FY 2012 NCE examinees indicated that they scheduled their examination on the web. The responses to the other six survey questions are summarized in Table

16 TABLE 18 SUMMARY OF RESPONSES TO SURVEY QUESTIONS SCHEDULING AND REGISTRATION COUNT PERCENT I WAS ABLE TO SCHEDULE AN ACCEPTABLE TEST DATE. AGREE % DISAGREE % I WAS ABLE TO SCHEDULE AN ACCEPTABLE TEST CENTER LOCATION. THE EXAM RESERVATION PROCESS WAS EASY TO USE. AGREE % DISAGREE % AGREE % DISAGREE 2 0.5% THE TEST CENTER WAS EASY TO LOCATE. AGREE % DISAGREE 6 1.4% THE TEST CENTER STAFF WAS HELPFUL AND KNOWLEDGEABLE. AGREE % DISAGREE 2 0.5% THE TESTING CENTER REGISTRATION/CHECK-IN PROCESS WAS HANDLED IN A PROFESSIONAL AND EFFICIENT MANNER. AGREE % DISAGREE 3 0.7% The next nine statements relate to examination experience, such as question fairness and readability of the examination. Table 19 summarizes the responses to these survey questions. Overall (98.6% agreement), the FY 2012 NCE examinees were satisfied with their testing experience. 14

17 TABLE 19 SUMMARY OF RESPONSES TO SURVEY QUESTIONS: EXAMINATION EXPERIENCE I THOUGHT THE EXAMINATION QUESTIONS WERE FAIR. COUNT PERCENT AGREE % DISAGREE % THE GRAPHS AND DIAGRAMS WERE EASY TO READ. AGREE % DISAGREE % THE GRAPHS, FIGURES AND DIAGRAMS IN THE QUESTIONS FIT ONTO THE SCREEN. AGREE % DISAGREE % I WAS ABLE TO "SCROLL" IN ORDER TO VIEW AN ENTIRE GRAPH OR FIGURE IN A QUESTION. THE AREAS OF THE CONTENT OUTLINE WERE FAIRLY REPRESENTED. AGREE % DISAGREE % MY TESTING ENVIRONMENT WAS CLEAN, QUIET AND COMFORTABLE. AGREE % DISAGREE % AGREE % DISAGREE 4 1.0% I ENCOUNTERED NO TECHNICAL PROBLEMS WITH THE TEST ADMINISTRATION SOFTWARE. THE TEST ADMINISTRATION SOFTWARE WAS USER-FRIENDLY AGREE % DISAGREE % AGREE % DISAGREE 5 1.2% OVERALL, I WAS SATISFIED WITH MY TEST EXPERIENCE AGREE % DISAGREE 6 1.4% 15

18 Since August 2009, the NBCRNA has administered alternative question formats (AQF) on the NCE. These question formats included multiple correct response (MCRs; where the examinee is asked to select an indicated number of correct responses), short answer / calculation (SA; where the examinee types in a short, numerical response), drag and drop (used for matching or ordering questions) and hotspot (in which the examinee identifies and clicks a particular visual target). A random sample (N=1,295) of examinees was given the opportunity to record their feedback on these question formats. Table 20 summarizes the responses to six survey questions relating to the MCR, SA, Drag and Drop, and Hotspot question formats. TABLE 20 SUMMARY OF RESPONSES TO SURVEY QUESTIONS: ALTERNATIVE QUESTION FORMATS COUNT PERCENT THE QUESTIONS IN THE MULTIPLE CORRECT RESPONSE (MCR) FORMAT WERE FAIR. AGREE % DISAGREE % THE QUESTIONS IN THE SHORT ANSWER / CALCULATION FORMAT WERE FAIR. AGREE % DISAGREE % THE QUESTIONS IN THE DRAG AND DROP FORMAT WERE FAIR. THE QUESTIONS IN THE HOTSPOT FORMAT WERE FAIR. AGREE % DISAGREE % AGREE % DISAGREE % I UNDERSTOOD HOW TO RESPOND TO THE QUESTIONS IN THE AQF FORMATS. AGREE % DISAGREE % I NEEDED HELP FIGURING OUT HOW TO RESPOND TO THE QUESTIONS IN THE AQF FORMATS. AGREE % DISAGREE % 16

19 The final three statements of the exit survey, asking about exam preparation, are summarized in Table 21. Of the FY 2012 NCE examinees, 52.9% stated the SEE helped prepare them for the NCE. A small portion (8.2%) of the FY 2012 sample indicated that they did not take the SEE. The majority (93.0%) attended a review course while only 6.9% did not. Finally, over two-thirds of the sample of examinees reported that their nurse anesthesia educational program featured the use of computerized testing. TABLE 21 SUMMARY OF RESPONSES TO SURVEY QUESTIONS: EXAM PREPARATION COUNT PERCENT TAKING THE SEE HELPED PREPARE ME TO TAKE THE NCE. AGREE % DISAGREE % DID NOT TAKE SEE % IF YOU TOOK A REVIEW COURSE IN PREPARATION FOR THIS EXAMINATION, PLEASE INDICATE BELOW WHICH REVIEW COURSE YOU TOOK. VALLEY ANESTHESIA % CORE CONCEPTS % HOWARD REVIEW 5 0.4% R&R BOARD REVIEW % PRODIGY ANESTHESIA % CRNA SECRETS 1 0.1% AANA ANNUAL MTG % OTHER COMMERCIAL % CRNA PROGRAM % DID NOT TAKE % PLEASE INDICATE BELOW IF YOUR NURSE ANESTHESIA EDUCATIONAL PROGRAM FEATURED ANY ACADEMIC TESTS USING COMPUTER BASED TESTING. YES % NO % 17

20 6. Demographic Characteristics of FY 2012 SEE Test Taker Population The following tables summarize FY 2012 performance on the SEE by gender, age, clinical background, and degree. Within each demographic, the scores are also broken down by year in program. The mean total score for Year 2 examinees (402.8) was higher than the mean total score for Year 1 examinees (386.6). Also, summaries of SEE total scores and subscores can be found in Table A3 of the Appendix. Table 22 summarizes SEE scores by gender: 37.5% of SEE examinees were male, 62.5% female. In general, males scored slightly higher on the exam. TABLE 22 FY 2012 SEE TEST TAKER PERFORMANCE BY GENDER AND YEAR IN PROGRAM YEAR 1 GENDER COUNT MEAN STANDARD DEVIATION FEMALE MALE TOTAL YEAR 2 FEMALE MALE TOTAL TOTAL FEMALE MALE TOTAL

21 Table 23 summarizes SEE scores by age group. The average age of Year 1 SEE examinees was 31.2 years (about the same, on average, as Year 1 examinees in calendar year 2011). The average age of Year 2 SEE examinees was 32.1 years. The mean age of all FY 2012 SEE examinees was 31.9 years, on average about 1 year younger than the sample of first-time NCE examinees in FY The largest subgroup (by age) was composed of examinees under the age of 30 (44.0%). Examinees between the ages of 30 and 35 comprised a slightly smaller subgroup (32.7%). There was a higher prevalence of under-30 examinees in Year 1 (48.3%) than Year 2 (43.0%). In general, younger examinees tended to score higher on the SEE than older examines. TABLE 23 FY 2012 SEE TEST TAKER PERFORMANCE BY AGE AND YEAR IN PROGRAM YEAR 1 AGE COUNT MEAN STANDARD DEVIATION UNDER AND OVER TOTAL YEAR 2 UNDER AND OVER TOTAL TOTAL UNDER AND OVER TOTAL

22 Table 24 displays the summaries of SEE scores by clinical background. TABLE 24 FY 2012 SEE TEST TAKER PERFORMANCE BY CLINICAL BACKGROUND AND YEAR IN PROGRAM YEAR 1 CLINICAL BACKGROUND COUNT MEAN STANDARD DEVIATION CCU ER ICU/CCU MICU NEURO ICU NICU OR PACU PICU SICU Trauma ICU TOTAL YEAR 2 CCU ER ICU/CCU MICU NEURO ICU NICU OR Other PACU PICU SICU Trauma ICU TOTAL

23 TABLE 24, CONT D YEAR 1 AND YEAR 2 TOTAL CLINICAL BACKGROUND COUNT MEAN STANDARD DEVIATION CCU ER ICU/CCU MICU NEURO ICU NICU OR Other PACU PICU SICU Trauma ICU TOTAL

24 Tables 25 displays summaries of SEE scores by degree to be attained. Over half (55.1%) of the FY 2012 SEE examinees were enrolled in the MSN degree programs. A small but growing number of SEE examinees (82) were students coming out of Doctorate-level programs. This number represents 2.3% of the entire FY 2012 SEE testing population. Score comparisons made among groups represented in these tables should be made with caution because some of the subgroups reflect small sample sizes. TABLE 25 FY 2012 SEE TEST TAKER PERFORMANCE BY TYPE OF GRADUATE DEGREE AND YEAR IN PROGRAM YEAR 1 DEGREE UPON COMPLETION COUNT MEAN STANDARD DEVIATION MS Nurse Anesthesia MS Nursing Major Other Masters TOTAL YEAR 2 MS Nurse Anesthesia MS Nursing Major Other Masters Post Masters Certificate Doctoral TOTAL TOTAL MS Nurse Anesthesia MS Nursing Major Other Masters Post Masters Certificate Doctoral TOTAL

25 Appendix TABLE A1 NCE PASS RATE TRENDS- FIRST TIME NCE TAKERS 2007 FY 2012 ADMINISTRATION DATE(S) PERCENT PASS (graduates ) (graduates after 2008) FY TABLE A2 DESCRIPTIVE STATISTICS FY 2012 NCE SCORES - FIRST TIME NCE TAKERS AVERAGE SD Total Score Basic Sciences Equipment, Instrumentation & Technology Basic Principles Advanced Principles Professional Issues TABLE A3 DESCRIPTIVE STATISTICS FY 2012 SEE SCORES 3 1 ST YEAR IN PROGRAM 2 ND YEAR IN PROGRAM ALL AVERAGE SD AVERAGE SD AVERAGE SD Total Score Professional & Legal Aspects Anatomy, Physiology & Pathophysiology Pharmacology Basic Principles Advanced Principles The policy limiting the time frame for passing the National Certification Examination (NCE) to 2 years following completion of a nurse anesthesia educational program was implemented. This was the last opportunity for a number of first time candidates who had completed programs prior to 2007, but who had never tested, to take the NCE. The reporting data for those few first time takers was separately identified. 3 For more information on SEE comparative data, please see the SEE Interpretive Guides under Certification > SEE Resources or Certification > Program Administration > Reports at 23

26 3,000 2,500 2,000 # Graduated # Certified 1,807 1,778 1,969 2,242 2,006 2,039 2,035 2,335 2,357 2,322 2,210 2,492 2,394 2,493 1,500 1, FY 2006 FY 2007 FY 2008 FY 2009 FY 2010 FY 2011 FY 2012 Figure A1. Number of Nurse Anesthetists Graduated and Certified by Year 24

27 Figure A2. Record of Clinical Experience Form 25