laboratorymedicine> august 2001> number 8> volume 32

Transcription

1 interpretation [management/administration and training generalist] A Four-Part Approach to Competency Assessment Jennifer Schiffgens, MBA, MT(ASCP), 1 and Valerie Bush, PhD, MT(ASCP) 2 From 1 Clinical Laboratory Improvement Consultants, Chicago, IL, and 2 Becton Dickinson, Franklin Lakes, NJ Competency programs must assess all core competencies and address employee preparation, theoretical testing, practical examination, and postevaluation follow-up. Competency programs must be consistently applied to both technical and nontechnical staff and should take an educational rather than a disciplinary approach. Clinical Laboratory Improvement Amendments (CLIA) 88 mandate that clinical laboratories administer competency testing to all employees annually. Specifically, the amendments state, The laboratory must have an ongoing mechanism to evaluate the effectiveness of its policies and procedures for assuring employee competence and, if applicable, consultant competence. 1 New employees must be assessed prior to 6 months of service, and all employees must be assessed at least annually. Under CLIA 88, Health Care Financing Administration does not recommend how competency testing should be accomplished. However, the College of American Pathologists laboratory general proposed checklist includes a checklist question and a suggested list of competency measures. 2 The Commission on Office Laboratory Accreditation, an independent association for laboratory accreditation, also has guidelines and suggestions for meeting the CLIA requirement. 3 Competency is the ability to do a job correctly and safely and to recognize and solve minor problems without needing assistance. 4 The competency assessment for testing and support personnel should assess the following core competencies: technical skills, judgment and decision making, knowledge base, and communication. 5 The competency scheme should cover the critical steps in the preanalytic, analytic, and postanalytic processes; each individual who performs a step in the process, whether technical or clerical, should be assessed. Many laboratories do not consistently assess the competency of the nontechnical employees who are essential to providing quality results. Federal guidelines do not require the assessment of 431

2 432 Employee Name: Department: nontechnical staff; however, if performed incorrectly, nontechnical functions can have an adverse impact on patient care. For example, if a test is ordered incorrectly, the patient either will not be tested or will have to endure multiple procedures if the test needs to be repeated. Currently, competency testing takes many forms and is administered in atmospheres ranging from educational to punitive. The College of American Pathologists Q-Probes study of the tools used to assess laboratory competence indicated that there is no consistent method being used by most laboratories. The study found that most laboratories used more than 4 methods to assess competency in the clinical disciplines, but 2 or fewer methods were used in the assessment of anatomic pathology, cytology, and phlebotomy/specimen handling. In addition, adherence to the competency assessment plan ranged from 78.8% in cytopathology to 90.4% in transfusion medicine for technical employees. 6 The methods used to assess staff competency range from review of daily work, quality control (QC), and quality Pass / Fail (check one) TASK PASS FAIL Identified patient Patient preparation Used aseptic technique Applied tourniquet properly Performed venipuncture adeptly Used correct collection tubes/proper order Released tourniquet correctly/at proper time Postvenipuncture patient care provided Labeled tubes properly Completed paperwork Prioritized for testing (STAT labeling) Followed safety protocols Notes: Assessor Signature: Employee Signature: [F1] An example of a competency assessment checklist for phlebotomy. indicators (QI) to direct observation, written tests, and evaluation of proficiency testing performance. A combination of these methods is probably most valuable, but rarely are all of these techniques used within a single institution for every employee. The review of daily work, QC, and QI indicators as a competency assessment gives relevant information regarding technical skills, but it does not address all steps in the testing process and may not identify areas of concern for specific individuals. Proficiency testing (PT) is another measure of analytic performance, but because proficiency testing samples are not received or reported in the same manner as patient samples, the preanalytic and postanalytic steps are not evaluated. In fact, Boone et al 7 noted that performance on mailed PT (ie, PT from a proficiency testing provider) was better than on blind PT (ie, in-house or shared-sample proficiency testing), suggesting that good performance on mailed PT did not necessarily indicate good performance on patient specimens. Written tests and direct observations of critical steps in the testing process address other core competencies. Written tests assess the individual s knowledge, and test-taking mimics the decision-making process. 8 Direct observation, including real-time review of QC, enables assessment of technical skills, judgment, and analytic decision making, but does not enable evaluation of the individual s knowledge base or advanced problemsolving capabilities. Written testing and direct observations can be combined for a comprehensive evaluation, but may be considered threatening by the employee. Competency assessment must be administered in an educational and nonpunitive environment. The goal of the assessment is to identify areas for improvement. Errors should be treated as potential problems with the system rather than employee inadequacies. 9 Competency assessment is another form of quality control. It gives laboratory management staff the opportunity to identify policy or procedural errors, in addition to human error. The laboratory does not reprimand the liquid controls when they are out of range or terminate a procedure when there is a typographic error. Steps are taken to correct the problem. Employees should be assessed with the same approach. To effectively assess employee competency, the laboratory must develop a program that covers all aspects of the testing process, addresses all core abilities and knowledge bases, is consistently applied to all technical and nontechnical employees, and is administered in an educational environment. The following 4- part approach meets these requirements. Employee Preparation Prior to competency testing, educational materials that cover the theoretical and practical aspects of the processes and tests should be provided to the employee. The tested personnel must understand the theoretical aspects of each step in the process and have the practical knowledge to perform the test. The educational materials also provide the employee with an opportunity to prepare for the competency testing. By preparing, the employee reinforces the information and gains confidence, resulting in positive

3 Accreditation Program Requirements College of American Pathologists Laboratory General Checklist 1 (Proposed) Question: 01:55500 phase: II Has the competency of each person to perform his/her assigned duties been assessed? Commentary: 01:55500 phase: II The competency of each person to perform the duties assigned must be assessed following training, and periodically thereafter. Retraining and reassessment of employee competency must occur when problems are identified with employee performance. The training and assessment program must be documented and specific for each job description. Those activities requiring judgment or interpretive skills must be included. The records must make it possible for the inspector to be able to determine what skills were assessed and how those skills were measured. Some elements of competency assessment include, but are not limited to: 1. Direct observations of routine patient test performance, including patient preparation, if applicable, specimen handling, processing, and testing; 2. Monitoring the recording and reporting of test results; 3. Review of intermediate test results or worksheets, quality control records, proficiency testing results, and preventive maintenance records; 4. Direct observation of performance of instrument maintenance and function checks; 5. Assessment of test performance through testing previously analyzed specimens, internal blind testing samples, or external proficiency testing samples; and 6. Evaluation of problem-solving skills. Available at: Accessed June 14, From Rabinovitch A, ed. Laboratory Accreditation Program Checklists. Northfield, IL: College of American Pathologists; Reprinted by permission. Commission on Office Laboratory Accreditation Guideline 291 Does your director or technical consultant periodically evaluate personnel performance? This review must also include the person s ability to perform the tests they are assigned. Also include a review of laboratory consultants, if you use them. (Laboratory consultant as defined by CLIA 88) From COLA. 3 performance. Advance preparation also promotes an educational rather than a punitive environment, thus benefiting the employees and the laboratory. This is especially important for new employees who must familiarize themselves with new and different systems and assays. Theoretical Testing A written examination that assesses the basic knowledge, problem-solving, judgment, and decision-making competencies should be administered. Written tests should evaluate whether or not the employee understands the theory and/or operating principles behind methods, instruments, policies, and procedures and reinforce knowledge learned in training. Questions should address all aspects of the testing process, including laboratory safety. The examination may also include instrument readouts, case studies, or other scenarios that assess problem-solving skills. Tests should be designed with questions that, when incorrectly answered, suggest that the employee does not understand the underlying principles of the process. 2 If multiple-choice questions are used, at least 5 choices should be given to reduce the likelihood of accurate guessing. The exam administrator can then be confident that the individual being assessed is knowledgeable about the subject. Essay questions are useful in assessing communication skills as well as knowledge; however, scoring the results can be time-consuming. Practical Examination The employee should be directly observed or a practical examination administered to assess mechanical techniques, judgment, decision-making processes, and communication skills. This enables the competency administrator to determine if the employee appropriately follows procedural steps and understands the workings of the method or instrument. For example, is the employee adding the correct number of drops of antisera during a crossmatch? Is the pipetting correct? Are proper dilutions being made? Are the tubes placed on the instrument 433

4 Examples of Theoretical Exam Questions 1. Which vitamin interferes with the dipstick glucose test? a. Vitamin B 12 b. Vitamin C c. Vitamin E d. Niacin e. Beta carotene 2. Maintaining a centrifuge ensures proper separation of the water and cellular phases of blood. The quality control maintenance measure for speed is: a. RCF b. Meters/sec c. G force d. MPH e. RPM Case 1. Al Bumin was being treated with unfractionated heparin for deep vein thrombosis. Due to an error in orders, he had 2 sequential coagulation specimens without a change in his dosage or therapy. The aptt for specimen 1 was 45 seconds and that for the second was 88.5 seconds. You are suspicious that something may be wrong with one of the specimens, and you notice that one of the tubes is underfilled. Which specimen is incorrect and why? Case 2. Mrs Mycin brought her 2-year-old daughter, Vanco, to the emergency department. The child had a fever, and her head was inclining to the right. Dr Kildare ordered a CBC count, glucose, electrolytes, and CSF for cell count, protein and glucose determinations, and microscopy. Only 1 CSF tube was sent to the laboratory for analysis. The results were as follows: Hematology Findings WBC /L RBC /L Hematocrit 48% Hemoglobin 16 g/dl Platelets /L Chemistry Findings Na 142 mmol/l K 5.0 mmol/l Cl 107 mmol/l CO2 22 mmol/l Glucose 81 mg/dl Cerebrospinal Fluid (CSF) Findings Protein 20 mg/dl (reference range, mg/dl) Glucose 65 mg/dl (reference range, mg/dl) Gram stain Few gram-positive cocci/hpf Culture Pending Cell count 0-2/HPF Appearance Clear Are these results consistent with meningitis? If not, what is wrong and why? 434 Answers 1. B 2. E Case 1: Sample 1. The sample tube was underfilled, causing platelet activation and a decrease in the aptt. Case 2: The results are not consistent with meningitis. The gram-positive cocci findings indicate that the sample is contaminated, probably due to processing by the chemistry department before processing by the microbiology department.

5 correctly? Is the required maintenance performed? Have the employee explain what he or she is doing and why during the practicum. This enables assessment of the employee s understanding of the theory behind the method and of communication skills. A checklist of predetermined skills can be used to document the observation. For the administrator, the checklist can also serve as an outline of the tasks that need to be observed. Each task has a unique set of skills that must be assessed, so the test administrator should review the procedure and outline the critical steps before developing the checklist [F1]. Postevaluation Follow-Up After completing the assessment, the competency program administrator must document and retain the results of the competency assessment for compliance purposes and follow-up with the employee to review the results. Retention of competency records should be based on regulatory, accrediting agency, and organizational requirements. 10 If it is discovered that there are gaps in an employee s knowledge, the employee can undergo further training and retake those portions of the test. If the employee is unable to satisfactorily pass the assessment after reeducation and training, then further action should be taken. This might include supervisory review of work, reassignment of duties, or in serious situations, termination of employment. Employees who satisfactorily complete the assessment, performing within all expected parameters, should be congratulated and perhaps rewarded. This may take the form of a pat on the back, a certificate of achievement, or other rewards that are within personnel guidelines. Today s laboratories have far fewer professionally educated and trained medical technologists and technicians than previously. This is partially due to a shortage of qualified individuals, costcontainment initiatives, and the downgrading of testing personnel requirements under CLIA 88. Technologic advances provide systems for more accurate and reliable test results, but the quality of testing still relies on the quality of the individuals who are involved in the testing process. The laboratory must ensure that personnel are trained and competent so that patient care and quality is not compromised. Instituting comprehensive competency programs that address all phases of the testing process, assess all the core competencies, and are consistently applied to both technical and nontechnical staff will have a significant impact on the laboratory. This 4-step approach promotes an educational rather than a disciplinary environment, emphasizes the importance of accurate and reliable results, and meets all federal guidelines for competency testing. 1. Medicaire, Medicaid and CLIA programs: regulations implementing the Clinical Laboratory Improvement Amendments of 1988 (CLIA). 57 Federal Register 7184 (1992). 2. College of American Pathologists. Laboratory general checklist proposed. Available at: Accessed June 14, Commission on Office Laboratory Accreditation (COLA). Laboratory Accreditation Manual. Rev 6. Criterion number 291 (Quality Assurance). Columbia, MD: COLA; Ruthemeyer M. The competency question. Radiol Manage. 2000; 22(5): Harmening DM, Castleberry BM, Lunz MF. Defining the roles of medical technologists and medical laboratory technicians. Lab Med. 1995;26: Howanitz PJ, Valenstein PN, Fine G. Employee competence and performance-based assessment: a College of American Pathologists Q-Probes study of laboratory personnel in 522 institutions. Arch Pathol Lab Med. 2000;124: Boone DJ, Hansen HJ, Hearn TL, et al. Laboratory evaluation and assistance efforts: mailed, on-site and blind proficiency testing surveys conducted by the Centers for Disease Control. Am J Public Health. 1982;72: Verhoeven BH, Hamers JGHC, Scherpbier AJJA, et al. The effect on reliability of adding a separate written assessment component to an objective structured clinical examination. Med Educ. 2000;34: Galel SA, Richards CA. Practical approaches to improve laboratory performance and transfusion safety. Am J Clin Pathol. 1997;107:S43-S National Committee for Clinical Laboratory Standards. Training verification for laboratory personnel: approved guideline GP-21A. Wayne, PA: National Committee for Clinical Laboratory Standards;