Presents Highlights from the Jimmy A. Young Memorial Lecture. November 30, 2007 Orlando, Florida

Transcription

1 Presents Highlights from the Jimmy A. Young Memorial Lecture November 30, 2007 Orlando, Florida 1

2 A Dialogue on the Evolution and Rationale for the NBRC Credentialing System =================== Presenters Theodore Oslick, MD, FACP, FCCP NBRC President Robert C Shaw Jr, PhD, RRT NBRC Assistant Executive Director 2

3 Evolution of the credentialing system The NBRC was formed in 1960 to credential RRTs The AARC perceived the need for credentialing at a lower level to meet human resource needs in 1968 The AARC s Technician Certification Board administered its first entry-level examination in

4 Evolution The Entry Level Examination was transferred to the NBRC in 1974 with the first administration in 1975 Combine all credentialing efforts under one organization Avoid antitrust issues for the AAIT Disentangle membership and candidate eligibility issues 4

5 Evolution RRT examinations have been administered in two parts since inception in 1960 Written and oral Written and clinical simulation (1979) 5

6 Evolution Results of the AARC s role delineation study verified there were at least two levels of practice for respiratory therapy practitioners Jouett ML. Delineation of the roles and functions of the entry-level generalist respiratory therapy practitioner. Sub-project I, HRS Dallas, TX: American Association for Respiratory Therapy,

7 Evolution Veizaga et.al.. vs. NBRC, AARC, and a group of hospitals in Chicago Lawsuit was settled in 1980 after direct evidence of job predictability was demonstrated by a criterion- related validation study NBRC agreed to revalidate examination results regularly 7

8 The NBRC s Mission Credential individuals Verify credentials Support state licensure and professional regulation 8

9 Limitations The NBRC s role is to measure what practitioners do, not what people think or wish should be done NBRC examination content must be appropriate nationwide This is true even when some practitioners perform procedures on the fringe of the spectrum 9

10 Conceptual Model NBRC Respiratory Therapy credentialing program is based on a hierarchal examination structure 10

11 Conceptual Model Fair RRT content Core content that is fair to include on the CRT examination Other competencies some therapists perform Only a few people perform these tasks nationwide and / or Judged relatively unimportant or insignificant 11

12 Limitations Before credentialing programs change, consideration must be given to human resources required to deliver care effects on previously credentialed practitioners historical professional recognition licensure regulations testing guidelines and standards 12

13 State Credentialing All states except Alaska, Hawaii, and Minnesota license respiratory therapists at the Certified Respiratory Therapist level Minnesota uses NBRC Credential for registration Florida, Louisiana, Maine, New York, North Dakota, Tennessee and West Virginia grant licenses at two levels All licensure/registration matched to NBRC credentials 13

14 State Credentialing All states credentialing therapists currently use NBRC Examinations Depend on the NBRC for valid examination results 14

15 Success rates Only 80% of graduates pass the CRT on their first attempt Testing over a broader, more complex set of competencies would diminish the flow of graduates into the workforce Fewer than 50% of graduates achieve the RRT credential on their first attempt 15

16 First Time Success for Those Who Seek the RRT 1,000 CRT 797 WRRT CSE

17 Impact of Changes A significant decrease in the flow of respiratory therapists into the workforce likely would produce undesired effects Infringement into scope of practice State developed licensing examinations of entry-level competencies of varied quality with varied standards out of the profession s s control Elimination of reciprocity among states 17

18 Impact of Proposed Changes Many state licensure laws specify the CRT Examination for licensure Licensure laws could be opened to other changes Could allow others an opportunity to infringe on the respiratory therapy scope of practice Might even result in elimination of an existing licensure act 18

19 19

20 Credentialing Program Validation 20

21 Triangle of supporting evidence Reliability Test Score Use Criterion Content 21

22 Reliability Evaluate and enhance or sustain reliability Define item metrics associated with high reliability Field-test items in the candidate population Feed back item metrics to content experts; revise some items and use the rest Optimize test score precision at the start of each test cycle through purposeful item selection Select a passing point associated with high decision consistency Monitor the decision consistency statistic for each set of test scores 22

23 Criterion-Referenced Evidence Supervisor Rating Inventory Critical Content Test Correlation of Ratings and Test Scores 23

24 Content Evidence Organize an inventory of tasks that describes the breadth of potential content A task describes one competency Administer the inventory in survey format Use survey responses to select critical tasks Retain tasks performed by at least a majority of respondents that respondents found important 24

25 Purpose of a Job Analysis Job analyses provide definitive evidence suitable for a legal defense that content included in a credentialing examination is job relevant and fair may yield some useful information about the fit between a program of credentialing examinations and practice, but will not give definitive guidance 25

26 Agreement Statistics for Task Inclusion Between CRT and RRT examinations Studies Extent Significance / Importance Thresholds 50% Thresholds 50% Thresholds Task Count All Kappa values were statistically significant at the.0001 level 26

27 Discussion Varying methods were used across these studies The number of tasks has declined and task descriptions have become more general Sampling and survey distribution shifted to an opt-in model in 2002 Measurement scales a 3-point 3 extent scale in 1992; a 2-point 2 scale in 1997 and 2002 an importance scale in 1992 and 1997; a significance scale in

28 Discussion There are multiple alternative hypotheses that could be associated with task inclusions agreement results over 3 studies Therefore, the trend cannot be trusted to reflect the real agreement between respondents for entry and advanced level studies 28

29 Comment If one really wanted to measure the agreement trend, one would standardize methods over time Measuring agreement or trends therein is not the purpose of a job analysis; legal defensibility is the purpose The panel appointed by the board does the best it can each time to legally justify content for its examinations 29

30 2007 CRT and RRT Job Analyses 818 technical directors were given the opportunity to rate task criticality for entry and advanced level therapists in their institutions 95% gave two ratings for each task 5% only gave one set of ratings 3% advanced level 2% entry level 30

31 Staff Therapists in Acute Care Hospitals 120, , ,000 FTE 80,000 60,000 55,729 86,702 Vacant FTEs Projected FTEs 40,000 20,000-2,832 5, ,000 31

32 Jan 1, 2002 to Dec 31, ,334, 42% 15,820, 58% CRT RRT 32

33 18000 W 105 th St Olathe, KS Phone (913) FAX (913)