Washington State Department Of Health Home Care Aide Competency Evaluation Program. November 6, 2013

Transcription

1 Washington State Department Of Health Home Care Aide Competency Evaluation Program November 6, 2013

2 Prometric Overview + Wholly-owned subsidiary of Educational Testing Service (ETS) + Full service provider of Test Development, Test Administration, and Data Management Services + Global leader in technology-enabled testing and assessment services + Over 10 Million exams administered annually + Over 400 clients in the healthcare, government, academic, financial, professional, and IT segments 2

3 Creating the Home Care Aide Certification Exam + WADOH contracted with Prometric to develop and administer the home care aide certification exam to evaluate whether an applicant possesses the skills and knowledge necessary to practice competently. + Industry-standard processes and procedures are the foundation of the exams. + WA Subject Matter Experts participated in every step. Job Analysis Item Writing Test Specifications Item Review Exam Assembly Statistical Analysis Standard Setting 3

4 Testing Concepts REQUIREMENT VALIDITY RELIABILITY FAIRNESS STANDARDIZED TESTING DEFINITION The extent to which inferences and actions made on the basis of a set of scores are appropriate and justified by evidence. Does the test: measure what it says it measures? identify minimally qualified individuals? An indicator of the extent to which scores will be consistent across different administrations. The extent to which a product or service is appropriate for members of different groups, and the extent to which users of products or services are treated the same way, regardless of gender, race, ethnicity, and the like. A test designed in such a way that the questions, conditions for administering, scoring procedures, and interpretations are consistent and are administered and scored in a predetermined, standard manner. 4

5 The Home Care Aide Certification Exam Specifications + Written/Knowledge Exam: + 50 scored items + 10 pre-test items; 90 minutes to complete + 3 Content Domains Roles and Responsibilities of the Home Care Aide (19 items) Supporting Physical and Psycho-Social Well-being (22 items) Promoting Safety (9 items) + 4 th to 6 th grade reading level + Oral versions of the exam are available in all languages + Skills Demonstration Exam: + Hand Washing, Common Care Practices, and 3 additional skills out of remaining 14 + Approx. 30 minutes to complete (based on specific combination of skills) + Administered by an RN or experienced LPN 5

6 Written Exam Pass Rates All Languages Combined Written Exam Pass Rates 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% Jan Feb March April May June July Aug Sept Oct Nov Dec % 84.6% 76.7% 80.4% 74.3% 77.3% 75.9% % 78.3% 78.6% 78.5% 78.7% 79.9% 76.4% 80.2% 78.6% 80.6% 6

7 Written/Knowledge Exam Passing Rates by Language English Russian Vietnamese 17% 25% 83% Pass Fail 40% 60% Pass Fail 75% Pass Fail 5,537 total exams High Score: 100% Low Score: 48% 316 total exams High Score: 98% Low Score: 32% 129 total exams High Score: 96% Low Score: 34% Spanish Korean Chinese 31% 48% 69% Pass 52% Pass 56% 44% Pass Fail Fail Fail 156 total exams High Score: 98% Low Score: 36% 172 total exams High Score: 98% Low Score: 58% 94 total exams High Score: 96% Low Score: 56% 7

8 Currently Available Languages + Established Languages + English + Russian + Spanish + Vietnamese + Korean + Chinese + New Languages as of 9/1/13 + Arabic + Khmer + Lao + Samoan + Somali + Tagalog + Ukrainian 8

9 Skills Exam Pass Rates Month over Month 100% Skills Exam Pass Rate 80% 60% 40% 20% 0% Jan Feb March April May June July Aug Sept Oct Nov Dec % 44.0% 42.1% 50.2% 74.6% 70.4% 71.5% 75.2% % 81.1% 79.6% 80.5% 86.3% 84.0% 85.3% 85.9% 82.1% 85.5% 9

10 Pass Rate By Skill May 2012 Thru Oct % 90% 80% 70% 60% 50% 40% Skill Pass Rates Above 85% Skill Pass Rates Below 85% 10

11 Most Frequently Missed Check Points Medication Skill + Medication Skill + Does the candidate select the correct medication bottle(s) 22% of candidates miss this point Heaviest weighted checkpoint (if missed candidate will fail skill) + Does the candidate give the correct medication dose placed into hand from the cap of the medicine bottle without the candidate touching the medication 17% of candidates miss rate Not heavily weighted + Does the candidate cue the client to drink full cup of water 27% of candidates miss this point Not heavily weighted + Does the candidate ask the client or check if medication is swallowed or check that it is swallowed 16% of candidates miss this point Not heavily weighted 11

12 Most Frequently Missed Checkpoints Feeding Skill + Does the candidate ensure the client is sitting upright in chair before begins feeding + 18% of candidates miss this point + Heaviest weighted checkpoint (if candidate missed this point they fail the skill) + Does the candidate sit to feed the client + 19% of candidates miss this point + Not heaviest weighted point + Does the candidate offer to apply clothing protector before beginning skill and remove before completing procedure + 17% of candidates miss this point + Not heaviest weighted point 12

13 Exam Administration - Regional & In- Facility Test Sites + Regional Test Sites + Originally located throughout the state based on population density and a comparison to nurse aide testing locations + Additional sites added in response to stakeholder feedback + In-Facility Test Sites + Can be any training program conducting home care aide training throughout the state. + Facility provides space and supplies for candidates to test. Agreement and supply list posted to website Prometric provides the evaluator Facility schedules the test date Must have minimum number of candidates to host an event Regional candidates can be added to meet minimum event size 13

14 Regional Test Site Map 14

15 September 2013 Testing Calendar SUNDAY MONDAY TUESDAY WEDNESDAY THURSDAY FRIDAY SATURDAY Longview Mt Vernon Spokane Pasco Pot Angeles Tacoma Bellevue Mt Vernon Everett Federal Way Bellevue Bellevue Bremerton Longview Mt Vernon Federal Way Everett Spokane Pasco Vancouver Tacoma Vancouver Federal Way Bellevue Federal Way Bremerton Spokane Mt Vernon Mt Vernon Pasco Port Angeles Vancouver Lynnwood Federal Way Bellevue Federal Way Bellevue Everett Longview Spokane Lynnwood Spokane Tacoma Vancouver Yakima Everett Bellevue Ferndale Pasco Spokane Olympia Port Orchard Aberdeen Federal Way Lynnwood Pasco Yakima Aberdeen Everett Olympia Port Orchard Bremerton Federal Way Lewiston Olympia Pasco 14 Omak Wenatchee 21 Olympia Pasco 28 Olympia Everett 15

16 Total Test Volumes 1200 Test Volumes Jan Feb Mar April May June July Aug Sept Oct Nov Dec

17 Program Enhancements + Website Re-Design + Candidate and program documents posted to website for easy use Documents provided in multiple languages Critical skills highlighted on Skills Checklist Most frequently missed checkpoints shared with candidates and instructors General Instructions provided so candidates can better prepare for test day Detailed explanation of Common Care Practices skill as it relates to other skills + Providing Written, Oral and Skills exam in 12 foreign languages + Korean, Chinese, Russian, Spanish, Vietnamese, Arabic, Somali, Samoan, Ukrainian, Tagalog, Laotian, Khmer Ground Breaking: No other known state certification includes a practical exam given in any other language than English + Inclusion of English text along side the foreign language text to assist bilingual candidates when dialects may not match exactly 17

18 Program Enhancements + Expanded Regional and In-Facility Test Site (IFT) Channels + 21 regional sites, including recently recruited sites in Seattle, Ferndale, Tacoma and Yakima + 5 IFT Site Locations Carnation, Des Moines, Bellevue, Port Angeles and Hoquiam (New) + Online Application Launched 8/23 + Allows candidate applications to be processed more quickly and scheduled sooner with the auto schedule capability applicants have applied online + Auto-schedule feature will place candidates into next available appointment at preferred location 18

19 Program Enhancements + Bulk Payment + Training and testing fees believed to be a barrier to certification to prospective Home Care Aides + Solution needed to allow post-exam payment for first time testers who are SEIU members Live as of 10/28/2013 WADOH sends Prometric a file of eligible candidates Prometric receives and processes applications to test and the candidate management database checks the eligibility file Candidates located in the file are scheduled into next available appointment at requested testing location Candidates not in the file yet are placed in a pending status until eligibility is received and then scheduled to test + Temporary workaround was in place from 9/1/13 on while technology updates were made 19

20 Questions and Answers How can I become a subject matter expert and sit on one of the Prometric panels Currently we are not hosting any test development sessions. However when we do we will reach out to the industry to recruit SMEs. You can send your resume to Prometric and we will keep it on file. Just because you want to become an SME on one of our panels doesn't mean you will be selected. There are specific criteria that are used to help select the experts. Once those experts are selected we work with the Department of Health to select the final group. How many non-english exams have Prometric given and are they all home care aide exams? Prometric delivers exams all over the world for different programs, certifications and licensures in hundreds of different languages. DO LEP examinees get more time for the skills exam since everyone must be communicated through interpreters? No, candidates who need an interpreter do not get extra time for the skills exam. Year to date we have not had anyone not complete the skills exam in the allotted amount of time. The interpreter is used to communicate between the candidate taking the test and the evaluator observing the exam. This is done simultaneously while the exam is being administered. 20

21 Questions and Answers What are the topics most missed on the written test? How can we get feedback on which sections we need to focus on? On the candidates score report we provide the candidate with a diagnostic feed back result of how many questions were asked on each section and how many were answered correctly in each domain. The domain outline is printed in the candidate bulletin which is posted on the website. If the candidate wants to share that information with you they can. Additionally, we are currently working on providing the Department with reports that rates each candidates performance in each of the three domains. That report should provide a better understand on where candidates are missing on the written exam. Did you say there was a site which shows what knowledge or skills are a problem for candidates? The frequently missed checkpoints for candidates across the state are listed on the website. This document lists the frequently missed checkpoints by candidates when taking the skills exam. Also on the website is a list of the skill checkpoints for all possible skills and the heaviest weighted points for those skills. Is there any way we can have some choice in when people receive the test? We have struggled with having to pay the extra money to find a different test date when the assigned one doesn't work with an employee's schedule. Candidates can select their test site location but cannot select the test date. We do understand that in some cases candidates cannot make their scheduled exam date so they can change their exam date with us. Depending on when they contact us or in some cases how often they want to change their date the candidate can be charged a reschedule fee of $25. Most candidates are given a one time courtesy and the rescheduling fee is waived. I would like to see the Skills Checklist translated into the 12 languages. Prometric and the DOH are currently working together to provide the skills checklist in all 12 languages. As soon as the list has been translated the documents will be posted to the website. 21

22 Question and Answers Can the candidate start over in the skills exam as long as they stay within their time? As long as the candidate has not told the evaluator that they have completed the skill, the candidate can start over. If time expires the exam is over, the candidate cannot start over or continue with the skill demonstration. Does the testing results reflect first time testers or does it reflect pass rates based on multiple testing events? The test volume slides as well as the pass rate slides include first time and re-tester. Does all testing for HCA certifications for the State go through Prometric? Prometric is the sole source vendor and is responsible for delivering all of the HCA certification exams. Is there a cost to Prometric to secure testing sites? If so, would a discount be available to facilities that provide their own sites? There is a cost to Prometric when using regional test sites. Every time we host an event at a regional test site we have to pay the site owner the negotiated fee. If we host an event at an In-Facility test site there is no test site cost for Prometric. In-Facility testing is when a training program allows Prometric to come into their facility to test their candidates. The In-Facility site is set up just like one of the regional test sites but the only candidates that go there are the ones that belong to that facility/training program. 22

23 Questions and Answers What is the testing fee? $115 for a written and skills exam or $125 for a oral and skills exam A comment on the many students failing having the patient sitting upright for feeding: It is because the actor is already sitting upright? The most frequently missed checkpoint for this skill is make sure the client is sitting in an upright position. For this skill the evaluator positions the client actor in a position that has him/her slumped over in a chair and the candidate has to position the client actor in the upright position. Are there any testing sites in Yakima? Yes, please visit the regional test site listing on for an updated listing of regional test site locations. For online applications, is it now possible for myself as training coordinator to complete applications for my students, negating the need for signatures or paper applications? Yes, please use the Online Application to submit all applications. Signatures and course completion certificates are no longer needed when submitting the certification examination application to Prometric. How do we request new test sites? You can send an to daniel.samila@prometric.com and he'll get in touch with you to talk about the need. 23

24 Questions and Answers Is it possible to get a Home Care Aide Certification without a social security number? (People who come with Visas and such) When presented with a situation where an individual cannot obtain a SSN required to obtain a DOH credential and renewal. We offer an option of completing a waiver. This will only be for individuals who meet specific criteria, including exchange students; non-resident aliens who do not qualify for a SSN; or persons who are practicing out of the country. The credential may be approved without a SSN if one or more of the following criteria are met, in which the applicant informs us that he or she: + Has applied for and been denied a SSN by the SSA. The credential holder must provide a copy of all documentation related to the SSA s decision. + Does not have a SSN, but has an individual taxpayer identification number. + The applicant provides the individual tax number. + The individual taxpayer identification number is recorded in the ILRS contact notes field. + Is a foreign national with a student visa. The person does not qualify for a SSN because of their visa status. + Is a foreign national who is not currently practicing in the United States. The person does not qualify for a SSN. + Will be in the United States on a visa. The person cannot apply for a SSN until their visa is approved and they have entered the United States. + A credential will not be approved for SSN waiver if questionable issues exist that require further review by disciplining authority, including failure to provide a SSN. 24

25 Question and Answer + We send the following information to applicants who do not provide an SSN: + Washington State law requires applicants for professional licenses to provide social security numbers. (42 USC 666; RCW ) We do not accept U.S. Individual Taxpayer Identification Numbers (ITIN) or Canadian Social Insurance Numbers as substitutes. + I have not provided a social security number because: I do not have a social security number. When I applied for one after my health profession credential was issued, it was denied. (Send anything you received from the Social Security Administration.) I do not have a social security number. I have an individual taxpayer identification number, which is. (If you check this box, you must also check Other and provide an explanation as to why you still do not have a social security number.) I am a foreign national not practicing within the United States. I do not qualify for a social security number. Other (Provide an explanation or number if forgotten on original application) 25