A Roadmap for Competency Assessment ASCLS-NJ Spring Seminar April 24, 2014 Dickie Nichols, MT(ASCP)SBB Technical Marketing Manager with Regina Castor BS MT(ASCP) SBB cm Area Technical Consultant, East Immucor, Inc.
Objectives Discuss requirements related to competency assessment Differentiate between competency and proficiency Describe what assessors, inspectors, and surveyors look for as evidence of compliance Identify methods to meet the requirements for competency assessment
Competency vs. Proficiency Competency Individual level New employee twice per year Incumbent annually 6 CLIA elements per test Proficiency Lab level Minimally twice per year per test Every test in system Results only
Why Competency Assessment? Dedicated Staff Complacency SOP drift Rotating Staff Loss of familiarity Infrequent activities SOP /Methodology Changes Required by Federal regulations
Why Competency? CMS has alerted us its on their Radar CLIA website: www.cms.gov/clia http://www.cms.gov/regulations-and-guidance/legislation/clia/downloads/clia_compbrochure_508.pdf Regulatory Agencies are looking at your compliance with CLIA requirements
What is CLIA? Federal Standard Compliance required Federal payment Assessed through various means CMS currently grants deemed status to: CAP AABB TJC (Joint Commission) Others
CLIA Competency Assessment Key Requirement 493.1413(b)(8)(9) & 1451(b)(8)(9) Technical Consultant/Supervisor Responsibilities Evaluating the competency of all testing personnel and assuring that the staff maintain their competency to perform test procedures and report test results promptly, accurately and proficiently
CLIA Competency Assessment 6 assessment elements Must be performed for each test Must be performed for each tech who performs the test Must be performed twice during first year of performing the test Must be performed at least annually thereafter
CLIA: Element 1 Competency for all tests must include: Direct observations of routine patient test performance, including patient preparation, if applicable, specimen handling, processing and testing
CLIA: Element 2 Monitoring the recording and reporting of test results
CLIA: Element 3 Review of intermediate test results or worksheets, quality control records, proficiency testing results, and preventive maintenance records
CLIA: Element 4 Direct observation of performance of instrument maintenance and function checks
CLIA: Element 5 Assessment of test performance through testing previously analyzed specimens, internal blind testing samples or external proficiency testing samples
CLIA: Element 6 Assessment of problem solving skills
CAP GEN.55500 Competency Assessment The competency of each person to perform his/her assigned duties is assessed CLIA elements cited NOTE: The competency of each person to perform the duties assigned must be assessed following training before the person performs patient testing.
AABB STD 2.1.2 Training The blood bank or transfusion service shall have a process for identifying training needs and shall provide training for personnel performing critical tasks. TASKS NOT JUST TESTS!
AABB STD 2.1.3 Competence Evaluations of competence shall be performed before independent performance of assigned activities and at specified intervals.
What Do Assessors / Inspectors Look For? Training Competency Assessment
Road Map to Inspection Is there a policy process or procedure Is it adequate Is it being followed
3P s: Policy, Process or Procedure Addressing Training and Competency Laboratory General policies CLIA elements incorporated All tests/test systems Blood Bank SOPs specific for the testing performed
Compliance with Policy/Process/Procedure Who assessed competency? Was it defined? Assessors have to assessed What constitutes an assessment? Tools, Checklists Is it defined in the policy or procedure What tests are being evaluated? Must include all tests/systems every year Can t pick and choose How is it documented? Does practice match SOPs?
Beware of the Regulations! Testing Personnel All tests All CLIA elements MUST be used for evaluation Other Personnel Facility-specified
Considerations New employees Competency assessment separate from training Assessed twice in the first year Incumbents Annual assessment Documentation for staff that work on all shifts Tests being evaluated Is there any distinction made for testing that may be provided on day shift vs after hours? Special testing
Competent or Not? Does documentation show tech is competent? Minimum passing score defined? Statement or check-box indicating competence? If not competent, what was done? Does practice match 3P s?
I don t have time for this!!!
PART II Pathways
Route #1 Combine multiple elements into a single assessment
Let s Go Back to Basics Element 1: Direct observation of performance Element 2: Monitoring recording and reporting of test results Element 3: Review of worksheets, QC records, PT results, PM records Element 4: Direct observation of instrument maintenance/function checks Element 5: Testing of previously analyzed specimens Element 6: Assessment of problem solving skills
Route #2 SOP Checklist
Direct Observation Checklist Make a checklist from your SOP Use at conclusion of training on that SOP Use for 6 month/90-day competency assessment Annual competency assessment
Donor Center Audit Checklist: *PPE *Bag Inspected *Bag hung properly *Tubing inserted *Hemostat applied *Sample port remains below donor arm etc.
Issuing Blood
Initial Training
Route #3 Use Tracer Audits
Tracer Audits Effective at looking at a variety of documents, staff Can be Retrospective Following a unit being transfused back to time of collection, reagents used, QC, equipment used, results recording, etc. Can be forwarded looking Ex: following equipment from purchase to discontinuation Crossmatch sample collection through transfusion Donor collection through processing/labeling Tracer audits can easily incorporate several elements of CLIA by looking at a variety of records
Route #4 Use existing maintenance schedules to plan assessments
Let s Go Back to Basics Element 1: Direct observation of performance Element 2: Monitoring recording and reporting of test results Element 3: Review of worksheets, QC records, PT results, PM records Element 4: Direct observation of instrument maintenance/function checks Element 5: Testing of previously analyzed specimens Element 6: Assessment of problem solving skills
Instrument Maintenance Direct observation of performance of instrument maintenance and function checks Which instruments? Who does it?
Use QA schedules to determine who and what to observe
Route #5 Delegate Assessments
Who Can Assess? Someone who Has been trained Can determine competent vs non-competent behavior Is motivated to do it right Must meet CLIA requirements It Doesn t Always Have to Be the Supervisor!!
Who to Consider Secret Shoppers Lead Techs Techs trained to task Others
Just Remember Those who assess competency must also have their competency assessed IF they perform critical tasks! And That Includes the Supervisor!!
Route #6 Use tasks for assessment that are already being performed
Let s Go Back to Basics Element 1: Direct observation of performance Element 2: Monitoring recording and reporting of test results Element 3: Review of worksheets, QC records, PT results, PM records Element 4: Direct observation of instrument maintenance/function checks Element 5: Testing of previously analyzed specimens Element 6: Assessment of problem solving skills
Aren t You Doing This Already?! Include as part of the competency policy Include items already being reviewed: Recording and reporting of test results QC records Preventive maintenance records Worksheets Document!
All Routine Tasks 2011 Blood Bank Competency Assessment Summary 1. Direct Observation of routine patient test performance CLIA Element Tech 1 Tech 2 Tech3 Tech4 Tech5 Tech 6 Tech 7 ABO/Rh and 12/23/201 antibody screen 11/1/2011 11/1/2011 11/1/2011 11/1/2011 Sep-11 12/6/2011 1 Other Accession # of test reviewed 2. Monitoring, recording and reporting of test results CLIA Element Tech 1 Tech 2 Tech3 Tech4 Tech5 Tech 6 Tech 7 ABO/Rh 0223IH9 0223IH11 0207IH16 0225;IH10 0206IH13 12/6/2011 0225;IH30 Antibody Screen 0223IH9 0223IH11 0207IH16 0225;IH10 0206IH13 12/6/2011 0225;IH30 Compatibility 0223IH9 0223IH11 0207IH16 0225;IH10 0206IH13 12/6/2011 0225;IH30 Antigen Typing 2/26/11 6/30/11 1/18/11 2/25/11 7/23/11 NA NA Antibody ID 0105IH31 0811IH11 0131IH6 0801;IH21 NA NA TRXN 0317;IH16 0228IH24 Courtesy of Lea Tolzmann, Winter Haven Hospital, Winter Haven, FL
Route #7 Sample Resources
Let s Go Back to Basics Element 1: Direct observation of performance Element 2: Monitoring recording and reporting of test results Element 3: Review of worksheets, QC records, PT results, PM records Element 4: Direct observation of instrument maintenance/function checks Element 5: Testing of previously analyzed specimens Element 6: Assessment of problem solving skills
Element 5: Assessment of test performance through testing previously analyzed specimens, internal blind testing samples or external proficiency testing samples
Previously Analyzed Specimens Patient/Donor samples Unique Samples Routine Samples Variable results depending on storage and use Less Cost Large resource availability Hematology Chemistry Document Results
Internally Prepared Samples Doctored patient/donor samples Can you get enough sample for multiple techs? Stability and reproducibility Can be time consuming to prepare Less Cost Document results
Externally Prepared Samples Available from some Blood Suppliers as a value added service Purchased tech competency products (as opposed to Lab Proficiency products) Scalable CEUs offered Can be used for multiple techs Proficiency products- more flexibility on how used
How can these products be used? They do not have to be treated in the same manner as routine testing ( Lab Proficiency) Direct observation potential Can be used by staff who have not completed a wet challenge Can be ordered as needed when new staff are hired Can be used for initial competency assessment Can be ordered as needed when methods are changed Use as corrective action for Laboratory PT failures Can be used regardless of test methodology being used Use as corrective action for near-miss events, Root cause analysis, etc.
Proficiency Samples May be used to fulfill competency requirement All labs have Limited volume/stability Unknown results May not be available when competency assessment is needed CE credits may be available
Limitations When Using Proficiency Samples for Competency Testing Rotating Lab Proficiency surveys does not satisfy all 6 of the CLIA requirements Proficiency samples must be treated like routine patient/donor samples Samples cannot be shared until results are received
Documentation of Proficiency Testing as Competency
Route #8 Problem Solving
Let s Go Back to Basics Element 1: Direct observation of performance Element 2: Monitoring recording and reporting of test results Element 3: Review of worksheets, QC records, PT results, PM records Element 4: Direct observation of instrument maintenance/function checks Element 5: Testing of previously analyzed specimens Element 6: Assessment of problem solving skills
Assessment of Problem Solving Skills Written test or quiz Case Studies Scenarios: What would you do if? Staff narrative
Could include What if questions to satisfy Problem solving requirement
Unsuccessful Result Process for Remediation Actions to take Removing employee from testing until competency is demonstrated Documentation Recurrence Re-assessment Determination Completion at end of each assessment
Documentation Trackable and Traceable Checklist not sufficient Each assessment requirement must be documented Direct observation checklists Title and date of record review Title, date, sample ID if using PT Graded test/quiz Assessor name(s) and dates Employee name
For more information CLIA website: www.cms.gov/clia Regulations may be found at http://wwwn.cdc.gov/clia/regs/toc.aspx
THANK YOU!!
Contact Information Dickie Nichols dnichols@immucor.com Regina Castor rcastor@immucor.com