Data Quality Assurance Assessment of SMR00 and SMR01 Dental Data. Project Summary

Transcription

1 Data Quality Assurance Assessment of SMR00 and SMR01 Dental Data Project Information Background Project Summary The retrospective quality assurance work carried out by ISD is an essential element of information governance and supports the credibility of national patient based data by providing a national perspective on data quality. The Chief Dental Officer for Scotland expressed concern about the quality of certain dental data that is maintained by ISD. As a result, ISD were asked to assess the accuracy of Outpatient (SMR00) and Inpatient/Day case (SMR01) dental data on the national database. The study was carried out in the two dental hospitals in Tayside and Glasgow and the dental institute in Edinburgh as well as three large general hospitals with dental specialties for both outpatients and inpatient/day cases. The following sites were assessed: Aberdeen Royal Infirmary Crosshouse Hospital Dundee Dental Hospital Edinburgh Dental Institute Glasgow Dental Hospital and School Hairmyres Hospital (ARI) (Crosshouse) (DDH) (EDI) (GDHS) (Hairmyres) It is each individual hospitals responsibility to ensure that all data held by them is up to date and accurate. Accurate and timely information can help to give a clear picture of a hospital s activity: Number of patients seen in a specified period Number of procedures carried out in a specified period Main diagnosis requiring the patient s hospital attendance Due to the small number of hospitals visited, caution should be applied if attempting to use the figures from this report as a measure of data accuracy for dental specialties across Scotland. 1

2 Sampling and Assessment Breakdown of Sample ISD assessed a sample of SMR00 Outpatient Attendance and SMR01 General Acute Inpatient & Day case episodes. The SMR00 sample of 100 records for each hospital was selected by the following criteria for both New and Follow-up / Return : stratified by specialty o with a clinic date within the most recent complete three month range. The SMR01 sample of a maximum of 75 records per specialty for each hospital was selected by the following criteria: submitted with a discharge date within the most recent complete three month period. The assessment of the full sample was not always possible due to reasons such as: o Records were unavailable, possibly due to patients attending for treatment during the assessment. o Records were removed before they were assessed, possibly due to patients attending at short notice. o There was insufficient information in the record regarding the episode ISD planned to assess. See below for a breakdown of the sample sizes, for records that ISD planned to assess and those ISD were able to assess. Episodes Assessed There were 580 episodes assessed in the SMR00 sample (Table 1). There were 473 episodes assessed in the SMR01 sample (Table 2). Table 1 - Breakdown of Assessment by Specialty Specialties No. of episodes No. of episodes requested assessed Oral Surgery Oral Medicine Orthodontics Restorative Dentistry Paediatric Dentistry Total Table 2 - Inpatient & Day case Breakdown of Assessment by Specialty Specialties No. Of episodes No. Of episodes requested assessed Community Dental Practice Oral Surgery Paediatric Dentistry Total Dundee Dental Hospital do not treat patients on an Inpatient / Day case basis. 2

3 Clinical Coding Issues and Working Practices The accuracy throughout this report for clinical data items refers to 3-digit level accuracy. Dental Hospitals & Institutes ISD found that none of the staff with the responsibility for recording clinical information had received any formal clinical coding training. Registered General Nurses at GDHS check automatically generated coding using a limited coding list of one diagnosis and two procedures. Clinical information was not recorded for outpatient attendances. Dundee Dental Hospital treat only outpatients, inpatients / day cases are treated at Ninewells Hospital. Inpatients / Day cases ISD found the accuracy of main diagnosis coding for Inpatient / day case admissions to be above 90% for both Edinburgh Dental Institute and Glasgow Dental Hospital and School. No other diagnoses (up to five other diagnoses can be recorded) or procedures (up to three other procedures can be recorded) were recorded by the dental hospitals for Inpatients / Day cases. Glasgow Dental Hospital and School and Edinburgh Dental Institute erroneously submit SMR01 returns for patients who do not attend for admission. There were seven and six SMR01 returns submitted respectively. ISD found that diagnoses were not recorded for outpatient clinics. Main diagnosis is not a mandatory data item for completion for an outpatient attendance. For a breakdown of the outpatient coding accuracy please see table 9 on page 6. Hairmyres Hospital did not record outpatient procedures for any of the records assessed by ISD. Main procedure is mandatory where applicable and should have been recorded when appropriate. Inpatients / Day cases ISD found the coding of main procedure for inpatient / day case admissions to be above 90% for all of the hospitals assessed. For a breakdown of the Inpatient / Day case coding accuracy please see table 10 on page 6. ISD found the accuracy of main diagnosis for inpatient / day case admissions to be above 90% for Aberdeen Royal Infirmary and Crosshouse Hospital. Coathill Hospital book theatre time at Hairmyres Hospital for the treatment of their patients. All patients from Coathill Hospital are treated in the specialty Community Dental Practice on a day case basis. Due to a lack of understanding regarding the clinical terms in the International Classification of Diseases and Related Health Problems Tenth Revision (ICD10) the diagnosis recorded for all community dental patients was selected to reflect that the patient was attending for extraction of a tooth / teeth instead of the condition which caused the tooth / teeth to require extraction. For Hairmyres Hospital, with the exception of the specialty Community Dental Practice, ISD found the coding of main diagnosis for inpatient / day case admissions to be above 90% for Hairmyres Hospital. 3

4 Non-Clinical Data Items The following non-clinical data items were assessed. Please note that the figures contained in the tables are for the records assessed. Table 3 - Outpatient Non-Clinical Data Items Accuracy SMR00 Data Items Completed Completed 1 Information Status not Available 2 Correct CI 95% Correct Clinic Date M % (99,100) 4 - Specialty M % (100,100) - - Consultant / HCP Responsible 3 M % (90,95) 36 - Referral Reason O Date Referral Received X % (67,76) na Referral Type M % (98,100) 5 - Referral Source C % (84,90) (56 na ) Clinic Attendance M % (98,100) 5 - Availability Status Code X % (74,83) na 1 completed is where a data item has been legitimately left blank. This also includes where it was not applicable (na) for a data item to be recorded. These have not been included in the accuracy calculation 2 - Benefit of doubt given to the information recorded on the SMR return as insufficient information was available to prove otherwise these have not been included in the accuracy calculation 3 - ISD found a large variation in the availability of information on Consultant / HCP Responsible for care between the sites visited. C - Conditional on context i.e. this data item is not mandatory for Follow-up / Return Table 4 Inpatient / Day Case Non-Clinical Data Items Accuracy SMR01 Data Items Completed Completed 1 Information Status not Available* Correct CI 95% Correct Specialty M % (85,91) 58 - Consultant / HCP Responsible M % (74,81) Management of Patient M % (78,85) 86 - Admission Date M % (99,100) 1 - Admission Type M % (82,88) 71 - Admission Reason O % (97,100) Availability Status Code X % (94,99) na Waiting List Date X % (75,84) na Waiting List Type M % (96,99) 12 - Discharge Date M % (98,100) 4-1 completed is where a data item has been legitimately left blank. This also includes where it was not applicable (na) for a data item to be recorded. These have not been included in the accuracy calculation * - Benefit of doubt given to the information recorded on the SMR return as insufficient information was available to prove otherwise these have not been included in the accuracy calculation 95% confidence intervals 95% confidence intervals (CI 95%) are used to convey the effects of sampling variation on the estimated accuracy rate and cannot control for other non-sampling errors such as biases in audit design, conduct or analysis. For example if 100 different audit samples (each of the same size) were selected, then we would expect that for 95 times out of 100 the estimated accuracy rate for a data item (for example main diagnosis) would lie within the lower and upper confidence intervals, and for the other 5 times would lie outside the confidence intervals. 4

5 ISD found that for the 314 patients who should have been recorded as New, 268 (91%) had the correct date that the referral was received recorded. A further four were recorded within two days of the correct date. Two patients had a date recorded that was incorrect by more than one year. ISD found that for the 433 patients who should have been recorded as being on the Inpatient/Day Case True Waiting List, 275 (82%) had the correct waiting list date recorded. A further 15 were recorded within two days of the correct date. Two patients had a date recorded that was incorrect by more than one year. Table 5 - Outpatient Non-Clinical Data Items Correctly Recorded Dental Hospitals and Institutes SMR00 Data Items Status DDH EDI GDHS ARI Crosshouse Hairmyres Clinic Date M % % 94 99% 89 99% % 96 98% Specialty M % % % % % % Consultant / HCP Responsible M 26 68% 68 84% 67 96% 84 97% 97 99% 91 96% Referral Reason O Date Referral Received X % 10 15% % % 82 98% 20 83% Referral Type M % 90 98% 91 99% 88 99% % % Referral Source C % 78 95% % 68 76% 97 99% 60 61% Clinic Attendance M % 92 99% % % % 95 97% Availability Status Code X 92 90% 0 0% 0 0% 4 67% 62 84% 83 87% The accuracy figures in this table exclude records where a data item was legitimately not completed or was given benefit of doubt as there was insufficient information to prove otherwise. C - Conditional on context i.e. this data item is not mandatory for Follow-up / Return - - data item not recorded so unable to calculate accuracy. Table 6 Inpatient / Day Case Non-Clinical Data Items Correctly Recorded Dental Hospitals & Institutes SMR01 Data Items Status DDH EDI GDHS ARI Crosshouse Hairmyres Specialty M % % % % % Consultant / HCP Responsible M % % % % 67 74% Management of Patient M % 0 0% % % % Admission Date M % 84 99% % % % Admission Type M % % % % % Admission Reason O % % % Availability Status Code X % 0 0% 2 100% % 16 76% Waiting List Date X % 52 62% 49 96% 43 83% 86 91% Waiting List Type M % 75 88% % % % Discharge Date M % 84 99% 72 97% % 93 99% The accuracy figures in this table exclude records where a data item was legitimately not completed or was given benefit of doubt as there was insufficient information to prove otherwise. - - data item not recorded so unable to calculate accuracy. 5

6 Clinical Data Items Table 7 - Breakdown of Recording of Main Diagnosis / Procedure Inpatients / Day Cases Diagnoses Procedures Diagnoses Procedures Code recorded by Hosp and ISD - 28 (27) 473 (378) 387 (355) Omitted 1 code required % of accurate recording 0% 21% 80% 78% No code recorded by Hosp or ISD Total Figures in brackets are the number that were found to be correct 1 An omitted code is where ISD have found evidence to support the recording of a diagnosis / procedure, which was not recorded on the SMR return. Table 8 - Breakdown of Recording of Other Diagnoses / Procedures Inpatients / Day Cases Diagnoses Procedures Diagnoses Procedures Code recorded by Hosp and ISD 1-1 (1) 144 (122) 65 (54) Omitted 2 code required Total Figures in brackets are the number that were found to be correct 1 Where a clinical code was recorded by the Hospital on the SMR return and the use of this position was agreed by ISD. This does not mean that the other diagnosis / procedure was correct. 2 An omitted code is where ISD have found evidence to support the recording of another diagnosis / procedure, which was not recorded on the SMR return. Table 9 - Count of Correctly Recorded Procedures for Dental Hospitals SMR00 Data Items DDH EDI GDHS ARI Crosshouse Hairmyres No. % No. % No. % No. % No. % No. % Main Procedure 0 0% 0 0% 0 0% 25 83% 2 100% 0 0% Other Procedures 0 0% - 100% - 100% 1 33% - 100% 0 0% Diagnoses are not mandatory for SMR00 submissions and were not recorded by any of the sites visited during this project. Where the number correctly recorded is 0 and the %age is 0, this indicates that the hospital did not record the information correctly in any of the required records. Where the number correctly recorded is - and the %age is 100%, this indicates that there was no information held in any of the records assessed to support the recording of another diagnosis or other procedure. The hospital was correct in having left this data item blank for all the episodes assessed. Table 10 - Count of Correctly Recorded Diagnoses and Procedures for Inpatients / Day cases Dental Hospitals SMR01 Data Items DDH 1 EDI GDHS ARI Crosshouse Hairmyres 2 No. % No. % No. % No. % No. % No. % Main Diagnosis % 83 98% 68 91% % 20 21% Other Diagnoses % 0 0% 52 63% 69 58% 4 11% Main Procedure % 70 83% 71 95% % 93 97% Other Procedures % 0 0% 22 71% 30 79% 5 50% 1 - Dundee do not treat patients on an Inpatient / Day case basis. 2 Coathill Hospital use theatre time in Hairmyres Hospital for the treatment of their Community Dental patients. The low percentage of correct Main Diagnosis is due to a misunderstanding of ICD10 code definitions by Coathill Hospital. Where the number correctly recorded is 0 and the %age is 0, this indicates that the hospital did not record the information correctly in any of the required records. Where the number correctly recorded is - and the %age is 100%, this indicates that there was no information held in any of the records assessed to support the recording of another diagnosis or other procedure. The hospital was correct in having left this data item blank for all the episodes assessed. 6

7 Recommendations (please note that not all of the following recommendations apply to all of the hospitals in the study) To comply fully with the Data Protection Act (1998) hospital staff should ensure that all patient data is accurate and up-to-date at all times. SMR01 - Inpatient / Day case returns should not be submitted when a patient does not attend for admission (GDHS & EDI). All findings in this report should be considered during the local development of current systems or when implementing new local systems. The recording of Availability Status Code is classed as Mandatory Where Applicable. Recording of this data item can sometimes explain waiting times exceeding the guarantee. The relevant Availability Status Code should be recorded for any Outpatient or Inpatient / Day case where it is deemed applicable. Return should continue to be recorded as such until the Consultant in charge of the patients care states that they are being discharged from their care. The time elapsed between clinic appointments should have no bearing on this (DDH currently reverting outpatients to new if further treatment is not required within one year). Every effort should be made towards recording the correct Consultant / HCP responsible for care for an episode of care. This data item can be important for helping ensure clinical governance. Consideration should be given to the coding of all available clinical information for Outpatient clinics. Main procedure is mandatory where applicable and should always be recorded when appropriate. All relevant diagnoses and procedures should be recorded for Inpatient / Day case admissions. Clinical coding of Day cases should be carried out on the diagnoses at discharge. This will improve the richness and accuracy of the data and its usefulness in the future (EDI currently record the diagnosis given at the consultation visit). Remove any automatically generated referral received date for return outpatient clinics that is generated by systems. (The PIMS used by GDHS defaults the referral received date to the same date as the return outpatient clinic. Effectively this creates a zero days wait for all return outpatient clinics). ISD manage a website which explains the definitions of data items recorded on SMR returns. The address for this site is Improving awareness of the definition of data items will help to improve the accuracy of the information recorded. Staff responsible for clinical coding should be aware of the following o ISD have a Coding Advisory Service (CAS) Helpdesk which can be contacted regarding any coding issues. The CAS Helpdesk operates from Tuesday to Thursday between the hours of 9am and 5pm. The contact telephone number is (0131) o From 1 st April 2006 OPCS4.3 will replace OPCS4. This will contain a wider range of codes that will allow more accurate recording of Dental procedures. o ICD10 and OPCS4.3 coding manuals can be made available to staff whose responsibility it is to code diagnoses and procedures. o Coding guidelines are issued quarterly by ISD to ensure staff responsible for clinical coding are aware in any change to the coding rules within Scotland. All staff responsible for carrying out clinical coding should receive clinical coding training. The Clinical Coding Tutors at ISD can arrange training courses tailored towards coding dental specialty information. 7