How are NCLEX Questions Written? The NCLEX Test Development Processes

Transcription

1 How are NCLEX Questions Written? The NCLEX Test Development Processes Roseann Colosimo, PhD, MSN, RN University of Nevada Las Vegas Catherine Dearman, RN, PhD University of West Florida

2 Conflict of Interest and Disclosures Neither the planners or presenters indicated that they have any real or perceived vested interest that relate to this presentation.

3 Learning Objectives Conference attendees will be able to: Describe the process used to develop and maintain the currency and accuracy of the NCLEX Understand the Practice Analysis as a basis for NCLEX item development Identify the item development process and regulatory oversight Dispel some of the common myths associated with the NCLEX Engage with presenters in some relevant case studies.

4 Presentation Outline History of the NCLEX Overview of the NCLEX Development Process Entry-Level Nursing Practice Analysis NCLEX Test Plan Item Development and Regulatory Oversight NCLEX True or False NCLEX Resources Online

5 History of the NCLEX

6 Brief History of U.S. Nursing Licensure Testing 1941: The State Board Test Pool Examination (SBTPE) was administered in four U.S. states, with the National League for Nursing (NLN) serving as the test service. 1950: All 48 U.S. states and Washington DC adopted the SBTPE as the jurisdictional licensure examination. 1978: NCSBN was established and served as the test developer of SBTPE. 1982: The NCSBN Board of Directors (BOD) officially changed the name of the examination to the National Council Licensure Examination (NCLEX). 1986: The NCSBN BOD began the investigation of computerized adaptive testing (CAT). 1994: CAT was successfully implemented for the NCLEX nationwide.

7 Overview of the NCLEX Development Process 7

8 NCLEX Development Cycle Practice Analysis: Entry-Level Nursing Scope of Practice Administer the Operational NCLEX Develop the NCLEX Test Plan New Items Pretest Item Writing Boards of Nursing Regulatory Review Item Review Item Bias and Fairness Review

9 Entry-Level Nursing Practice Analysis

10 What is a Practice Analysis? A large scale survey of job incumbents on activities that are performed on the job. For the development of NCLEX, newly licensed nurses are surveyed on current entrylevel nursing practice. The results are used to guide content distribution of the client need categories on the NCLEX and its test plan.

11 Practice Analysis Methodology RN and PN/LVN entry-level practice analyses are conducted separately using the same methodology. All RN and PN practice analysis methodology is reviewed by three external experts prior to survey data collection. A panel of 9 to 15 nurse subject matter experts develops a comprehensive list of entrylevel RN or PN activity statements. The comprehensive list of nursing activity statements forms the basis of the practice analysis survey.

12 Sample and Respondents The practice analysis survey sample reflects the population of NCLEX-RN or NCLEX- PN candidates. Sampling follows entry-level nurse distribution across the U.S. Percentage of respondents by states was also similar to the nursing representation across the U.S. Each practice analysis survey is sent to entry-level nurses at the appropriate licensure level. The response rates for RN and PN practice analyses are approximately 20% and 15%, respectively.

13 NCLEX Test Plan 13

14 NCLEX Test Plan Basic Test Plan Available in September prior to the effective year Provides basic information and updated client needs category percentages Detailed Test Plan Available in December prior to the effective year Sample Items for all client need categories Item Writer/Item Reviewer/Nurse Educator Version Candidate Version

15 NCLEX Detailed Test Plan Three important purposes 1. Guide candidates preparing for the exam 2. Direction for item development 3. Facilitate classification of exam items

16 Effective April 1, 2016 to March 31, 2019

17 Effective April 1, 2014 to March 31, 2017

18 Item Development and Regulatory Oversight 18

19 Item Writers Qualified nurse volunteers Actively teaching entry-level nursing students Create items (test questions) based on the current NCLEX Test Plan Have not participated in other nursing licensure exam or preparatory exam development in the past two years

20 Item Reviewers Qualified nurse volunteers Actively working with entry-level nurses providing care to clients Review test items for accuracy, currency and entry-level appropriateness Have not participated in other nursing licensure exam or preparatory exam development in the past two years

21 Item Bias and Fairness Reviews All items are reviewed to ensure the exclusion of sensitive, unintentional biased language. Items are edited to remove words or phrases which may affect exam measurement.

22 Item Bias Reviews Criteria Stereotypes Inflammatory material Language Assumptions

23 Oversight Committee Review NCLEX Examination Committee (NEC) Comprised of nurse regulators in various jurisdictions Develops and maintains NCLEX policies Reviews and approves all NCLEX items before use in scoring Review recommendations from all NCLEX review committees and determines item dispositions

24 Boards of Nursing Regulatory Review Conducted biannually Review criteria Entry-level appropriateness Nurse practice act congruency Comments forwarded to the NEC

25 NCLEX True or False

26 True or False? #1 Recent changes in nursing practice are not tested on the NCLEX. It takes a while for the NCLEX to incorporate changes in practice False! Each NCLEX item undergoes a number of review processes to ensure currency Practice Analysis Item Review NCLEX Examination Committee Board of Nursing/Regulatory Body Review Ongoing item pool maintenance Immediate changes are reflected in an Item Pool Search to uncover items with outdated content.

27 True or False? #2 The computer randomly selects students for an exam with the maximum number of questions. False! All NCLEX-RN or NCLEX-PN exams administered match the respective test plan percentages. Pass/Fail determinations are not made until the approved NCLEX Test Plan has been met. Items are only administered as they are needed to determine candidate s competence. Unnecessary item exposures may pose security threats to the exam.

28 True or False? #3 The NCLEX is more difficult for candidates who speak English as a second language. False! The language used in an NCLEX item is consistent with verbiage an entry-level nurse is expected to know. NCLEX readability is initially established at the start of item development and is frequently reviewed to ensure it remains intact. NCLEX-RN no higher than 1300 Lexiles e.g., David Copperfield (1340 Lexiles) NCLEX-PN no higher 1200 Lexiles e.g., Little Women (1201 Lexiles)

29 True or False? #4 The computer will continue to choose questions on a content topic previously answered incorrectly. False! The computer is programmed to select items based on test plan content areas and item difficulty. All NCLEX exams administered match the test plan percentages. Over-sampling from any one content area affects exam reliability.

30 True or False? #5 The NCLEX contains questions outside of the entry-level nursing scope of practice. False! The NCLEX Test Plan is based on the analysis of entry-level practice as outlined in the triennial NCLEX Practice Analysis. In other words, the NCLEX is built on the current practice of entry-level nurses within the context of the nursing scope of practice the allowable duties in nursing.

31 NCLEX Resources Online

32 NCLEX Resources Online NCLEX Candidate Bulletin ( This resource serves as your guide from before to after the NCLEX, including important contact information, details on how to register, the NCLEX Candidate Rules and results processing. NCLEX-RN and NCLEX-PN Test Plans & Detailed Test Plans ( Learn more about the content and scope of the NCLEX by reviewing the basic and detailed versions of the NCLEX Test Plans. RN & LPN/VN Practice Analyses ( Discover the triennial studies that provide the basis for the NCLEX-RN and NCLEX-PN Test Plans. 32

33 Online Resources (Continued) NCLEX Using CAT Video ( Familiarize yourself and your students with how computerized adaptive testing (CAT) works. NCSBN Portal (portal.ncsbn.org/) Register for the NCLEX and sign into your account for scheduling and Quick Results. NCLEX Tutorial ( Experience the same tutorial that NCLEX candidates take at the beginning of their exam, including a look at the different item formats that they might see on their exam. 33

34 Online Resources (Continued) NCLEX Frequently Asked Questions ( Find answers to your questions about the NCLEX, including questions from candidates and/or about exam development, psychometrics, alternate item formats and more. Like us on Facebook and Follow us on Twitter! ( Receive NCLEX information, news and updates, and ask any questions you may have about the NCLEX program. 34

35 How Nevada Improved! 35

36 Nevada Education of nurses not usually thought of when thinking Las Vegas 36

37 Skinny on real Nevada A few facts We have lovely Communities, schools, Churches, great Parks Shrimp consumption is Nevada is 60,000 pounds per day higher than rest of country combined Only state to posses a complete skeleton (55 feet) of an Ichthyosaur (extinct marine reptile) 150 couples get married everyday in Las Vegas 85% of Nevada is Federally owned 37

38 Nevada Health 38 up from 45 Healthy People 2020 land of disparities and access issues Low per capita public health funding Low rate of high school graduation 17% of adults smoke 25% adults Obese Children in poverty 24% Increased our immunization compliance to 67% 38

39 Education Consultant Nevada State Board of Nursing 13 RN programs 35,000 registered nurses 3 LVN 800 practical nurses Large area, 2 major cities Las Vegas, Reno 39

40 Nevada response Nevada Regulations requires an 80% full year pass rate for all schools of nursing for first time takers Nevada was 46 out of 56 states and territories. Deans and Directors were demanding a different measure be used but agreed other measures would be more labor intensive for them ( not a good option) Nevada State Board of Nursing upheld 80% first time test taker full year score as required. 40

41 Nevada Nursing School Approval categories Provisional - New schools have 2 years after first class graduates to achieve Nursing programmatic accreditation and 80% first time full year pass rate. NSBN has granted extensions to schools make good progress but has accepted teach out plans from schools unable to achieve Fully approved schools need to maintain nursing programmatic accreditation and 80% first time taker full year pass rate. If below 80% one year submit plan, if below two years conditional status and must have survey and consultant help. 41

42 Precipitating Factors Some not all Lack of understanding of NCLEX- negative attitude Many Faculty have Nurse Practitioner background Many part time adjunct faculty teaching clinical Double number of nursing schools in 7 years with increased growth Faculty shortage ( better black dealer and bartender schools) Competition for clinical practice settings Increased rigor in nursing practice 42

43 Nevada Explosive growth in home sales before crash Ripe for explosion of technical schools moving into teaching nursing Increasing population Always ranking in numbers of nurses per 100,000 citizens Building hospitals Technical schools already existed for massage, medical assisting and criminal justice programs 43

44 The NCLEX Story year NCLEX pass rate- NV NCLEX pass rate -National Number RN Schools /9 below 80% /11 below 80% /12 below 80% /12 below 80% /14 below 80% /12 below 80%

45 Perspective for totals Candidates Total RN 229,459 (69.87%) RN First time 166,523 (81.78%) Repeat 62,936 (38.36%) U.S.-Educated 206,170 (75.15%) Internationally Educated 23,289 (23.11%) 45

46 Nevada Action Plan Regional workshops by NCSBN Highly encourage test item development classes for faculty Faculty development NSBN communicates test plan changes and dates Faculty want to improve- provide resources 46

47 Regional workshops NCSBN Workshops Dispel myths Who has high scores Associate degree programs or BSN?? - Explain development of exam - Explain how many of your peers are involved - Explain how your faculty can participate - Focus on test plan - Also great networking for schools to share resources 47

48 Test Item Development Help faculty to think critically Not just rely on test item banks Better analysis of test item banks 48

49 Faculty development Clinical Faculty workshops NSBN facilitated Learning theories Flipped classroom Engage the student Classroom control Command of classroom Students wide range 40 years ago everyone was white,female and 19 Today 19-60, every ethnic group, 10 %male and rising, Many students at some schools have English as second language 49

50 Communication Nudge faculty meeting to discuss NCLEX Require reports to include test plan and curriculum reviews 50

51 NSBN Board reports can support Faculty resources when administrations are reluctant 51

52 Anecdotal experience -Nevada Schools that fail have 1. leadership not experienced in nursing education 2. Admission criteria that do not support national research 3. Faculty who maybe excellent clinicians but not educators 4. Lack local faculty control over curriculum 52

53 Fully approved schools Proof in pudding Maintained an above 90% PASS RATE IN 2013 AFTER INCREASE IN PASSING SCORE The systematic program evaluation of curriculum Threading make sure content and test plan covered. 53

54 NSBN Collaboration, Collaboration Positive feedback to faculty Deans and Directors at Meetings Positive meetings and constant collaboration Board paid initial cost to of computer program to do clinical scheduling releasing more faculty to teach 54

55 55