CLINICAL AUDIT All aspects of radiological practice are amenable to audit. Note that Clinical Audit is one of the preferred options for trainees undertaking project 1. This module provides trainees with a definition of clinical audit, descriptions of the types of audit, steps in the audit process, key indicators in radiological practice and examples of audits of these key indicators. What is clinical audit? Clinical audit is a quality improvement process that seeks to improve patient care and outcomes through systematic review of care against explicit criteria and the implementation of change. Aspects of the structure, processes, and outcomes of care are selected and systematically evaluated against explicit criteria. Where indicated, changes are implemented at an individual, team, or service level and further monitoring is used to confirm improvement in healthcare delivery. The clinical audit process should ideally be owned by healthcare professionals: they carry out the audit, discuss results and make improvements to practice, etc. The Difference Between Audit and. Research Research Discovers the right thing to do A series of 'one-off' projects Collects complex data Experiment rigorously defined Often possible to generalise the findings Audit Determines whether the right thing is being done A cyclical series of reviews Collects routine data Review of what clinicians actually do Not often possible to generalise from the findings Accuracy of audit Audit is a sampling process and unlike research is not primarily designed to be statistically robust since it is carried out for the purpose of improving local quality of care rather than influencing others practice. When audit data is interpreted, any potential statistical weaknesses should be taken into account. Page 1
Types of audit Audit can be divided into three categories. 1. Structure audit. This looks at the systems within which we work, for example the management structure, accommodation, equipment, staffing and training. 2. Process audit. Examination of the processes involved in the delivery of care from initial referral to delivery of a radiological report including for example quality management of the processes, justification, waiting times and examination practices and protocols. 3. Outcome audit. Examination of the outcome or results of the delivery of care which may include medical outcomes and patient satisfaction. Key indicators in radiological practice Work Load Access-waiting times (metro vs. regional) and cancellations Time from attendance for procedure/imaging to delivery of report to referrer Time from transcription of report to delivery of report to referrer Hand off time between trainee writing a report and review of report by consultant Justification for prescribing procedures/imaging Records Delay or failure to obtain previous images/ patient records Critical events in diagnostic radiology include but are not limited to: Films per examination, film reject rate Lost films, reports Unplanned / repeat imaging Unnecessary Imaging Radiation Dose Exposure Radiation does for CT Scans (Emergency Trauma Patients, Paediatric Patients) Diagnostic accuracy Reactions to contrast media Pneumothorax after Biopsy Infection from insertion of central line Contrast extravasation from CT scan injections Hematoma from lumbar punctures Adequate tissue sampling (does having a Cytologist on site make a difference?) Page 2
Steps in the Audit Process 1. Identify the need for change (structure, process, outcome) 2. Setting criteria and standards Setting criteria and standards entails stating what should be happening. A Criterion is an item of care or some aspect of care that can be used to assess quality. The criterion is written as a statement. Criteria can be defined from recent medical literature, and the best experience of clinical practice these are called 'Normative criteria'. To make the criteria (statement) useful the Standard needs to be defined. A Standard describes the level of care to be achieved for any particular criteria. Standards must be set. There are basically 3 options: A minimum standard. This describes the lowest acceptable standard of performance. Minimum standards are often used to distinguish between acceptable and unacceptable practice. An ideal standard describes the care it should be possible to give under ideal conditions, with no constraints. Such a standard by definition cannot usually be attained. An optimum standard lies between the minimum and the ideal. Setting an optimum standard requires judgment discussion and consensus with other members of the primary care team. Optimum standards represent the standard of care most likely to be achieved under normal conditions of practice. 3. Collecting data on performance Identify what data needs to be collected, how and in what form it needs to be collected, and who is going to collect it. Remember only collect information that is absolutely essential. 4. Assess performance against criteria and standards With the information collected analysis is possible, and identification of any area of care below the predetermined standard of the criteria can be made. The results can then be used to develop an action plan i.e. what needs to be done, how it needs to be done, who is going to do it and when is it going to be done. 5. Analysis of audit results Audit is primarily a quality improvement tool, and in those cases where the chosen standard is not reached, the results should be interpreted in a culture which does not seek to blame individuals. Analysis of the results should examine all the possible reasons for the results not meeting the standard, including the Page 3
target level chosen, system, process, and technical reasons. Only then can system changes be introduced to address any measured shortcomings. Consideration should also be given to possible sampling bias accounting for the under-performance. A checklist of suggested changes to improve performance should then be drawn up and implemented if deemed appropriate by the Head of Department. Re-audit The audit cycle is now almost complete, but without re-evaluating the care the practice is giving, it is impossible to see if recommendations have been implemented and the level of care improved. When change has been implemented it is mandatory to repeat the same audit process to ensure that the changes introduced have led to the expected improvement. This closes the loop. Examples of audits 1. Structure audit- type: staff training Standard: 100% of department staff should have completed training in cardiopulmonary resuscitation. Criterion to be measured: percentage of staff who have completed training within the timeframe specified in local rules. Data to be collected: the total number of staff, and the number who have undergone training. Suggested number of staff to be sampled: all staff. 2. Process audit- type: patient consent Standard: for 100% of interventional vascular radiology procedures there is documented evidence that a discussion of the procedure by a suitably qualified member of staff has taken place and there is a written record of patient consent. Criterion to be measured: the percentage of patients for whom there is evidence that consent procedures have been completed. Data to be collected: consecutive patient records examined for written evidence of pre-procedure discussion, the name of the doctor and the patient s written consent. Suggested number of patient records to be sampled: 30 consecutive interventional procedures. Page 4
3. Outcome audit- type: procedure complication rate Standard: fewer than 20% of lung biopsies should result in pneumothorax and fewer than 8% of patients should require chest drain insertion. Criterion to be measured: The percentage of patients who suffered a pneumothorax and the percentage requiring a chest drain. Data to be collected: consecutive lung biopsies, patient identifier, name of doctor, needle size used, presence or absence of pneumothorax, chest drain required or not. Suggested number of procedures to be sampled: all lung biopsies carried out in one year. Remember: when constructing Criteria and Standards Make unambiguous statements Keep the task focused on the audit project Refer to the literature indicating current practice Choose criteria and standards in line with current practice Ensure the criteria and standards are based on fact. Page 5
References Used in the Development of the Clinical Audit Module Clinical Audit European Society of Radiology Acknowledgement: Paper prepared by the ESR Subcommittee on Audit and Standards. Chairperson: Jane Adam. Members: Hudaver Alper, Éamann Breatnach, Maurizio Centonze, Elisabeth Dion, Birgit Ertl-Wagner, Robert Manns. Approved by the ESR Executive Council, June 2009. http://www.i3- journal.org/cms/website.php?id=/en/index/read/clinical-audit.htm NHS, National Institute for Clinical Excellence http://www.pdptoolkit.co.uk/files/guide%20to%20the%20pdp/new%20index/audit.htm Page 6