Camden and Islington NHS Foundation Trust Findings and Recommendations from the 2014/15 NHS Quality Report External Assurance Review Report to the Council of Governors Report: 14 May 2015
Contents Executive Summary 3 Executive Summary Performance Indicator testing 6 Care programme approach 7 day follow up 9 11 Access to crisis resolution home treatment team (gatekeeping) Local Indicator: Patient safety incident relating in severe harm or death Recommendations for improvement This report sets out the findings from our work on the 2014/15 Quality Accounts. We would like to take this opportunity to thank the management team for their assistance and co-operation during the course of our review 14 Recommendations for improvement 19 Update on prior year recommendations Appendices 22 Purpose of our report and responsibility statement 24 Data quality framework Delivering informed challenge Providing intelligent insight Growing stakeholder confidence Building trust in the profession 1
Executive Summary 2
Executive Summary We have completed our Quality Report testing and are in a position to issue our limited assurance opinion. Status of our work Receipt of the final signed Quality Report and letter of Representation. Completion of our Content and Consistency review as stakeholder comments are outstanding. Sample testing of day zero follow ups included in the calculation of the care programme approach 7 day follow up performance indicator. See page 7 for further information. Subject to the satisfactory completion of the above matters we anticipate signing an unmodified opinion for inclusion in your 2014/15 Annual Report. Q4 Governance Risk Rating: Green The Care Quality Commission did a full inspection of Camden and Islington in August 2014, they identified improvements needed. 2014/15 2013/14 Length of Quality Report 54 pages 56 pages Quality Priorities 7 4 Future year Quality Priorities 4 7 Scope of work We are required to: Review the content of the Quality Report for compliance with the requirements set out in Monitor s Annual Reporting Manual ( ARM ). Review the content of the Quality Report for consistency with various information sources specified in Monitor s detailed guidance, such as Board papers, the Trust s complaints report, staff and patients surveys and Care Quality Commission reports. Perform sample testing of three indicators. The Trust has selected 7 day follow up and Access to Crisis Resolution Home Treatment Team (Gatekeeping) as its publically reported indicators the alternative was Delayed Transfer of Care. For 2014/15, all Trusts are required to have testing performed on a local indicator selected by the Council of Governors. The Trust has selected Patient Safety Incidents resulting in Severe Harm or Death. The scope of testing includes an evaluation of the key processes and controls for managing and reporting the indicators; and sample testing of the data used to calculate the indicator back to supporting documentation. Provide a signed limited assurance report, covering whether: Anything has come to our attention that leads us to believe that the Quality Report has not been prepared in line with the requirements set out in the ARM; or is not consistent with the specified information sources; or There is evidence to suggest that the 7 day follow up and Access to Crisis Resolution Home Treatment Team indicators have not been reasonably stated in all material respects in accordance with the ARM requirements. Provide this report to the Council of Governors, setting out our findings and recommendations for improvements for the indicators tested: 7 day follow up, Access to Crisis Resolution Home Treatment Team and Patient Safety Incidents resulting in Sever Harm or Death. 3
Executive Summary (continued) We have not identified any significant issues from our work. Content and consistency review Review content Document review Interviews Form an opinion We have completed our content and consistency review, subject to our review of stakeholder comments. From our work, nothing has come to our attention that causes us to believe that, for the year ended 31 March 2015 the Quality Report is not prepared in all material respects in line with the criteria set out in the ARM. Overall conclusion Content Are the Quality Report contents in line with the requirements of the Annual Reporting Manual? Consistency Are the contents of the Quality Report consistent with the other information sources we have reviewed (such as Internal Audit Reports and reports of regulators)? Performance indicator testing Interviews Identify potential risk areas Detailed data testing B B Identify improvement areas Monitor requires Auditors to undertake detailed data testing on a sample basis of three mandated indicators. We perform our testing against the six dimensions of data quality that Monitor specifies in its guidance. From our work, nothing has come to our attention that causes us to believe that, for the year ended 31 March 2015, the indicators in the Quality Report subject to limited assurance have not been reasonably stated in all material respects in accordance with the ARM and the six dimensions of data quality set out in the Detailed Guidance for External Assurance on Quality Reports 2014/15. Accuracy Is data recorded correctly and is it in line with the methodology. Validity Has the data been produced in compliance with relevant requirements. Reliability Has data been collected using a stable process in a consistent manner over a period of time. Timeliness Is data captured as close to the associated event as possible and available for use within a reasonable time period. Relevance Does all data used generate the indicator meet eligibility requirements as defined by guidance. Completeness 7 Day follow up Gatekeeping Patient Incident Is all relevant information, as specific in the methodology, included in the G B G calculation. Recommendations identified? 4 4 4 Overall Conclusion A [Unmodified Opinion] B Unmodified Opinion B No opinion G No issues noted B Satisfactory minor issues only A Requires improvement R Significant improvement required B A B G B G G G G G B G G G G required 4
Performance indicator testing 5
Care programme approach 7 day follow up The Trust has a process for managing and monitoring this indicator with an internal target of 72-hour follow-up. Trust reported performance Target 2014/15 96.68% 95% 2013/14 97.35% 95% Overall evaluation Indicator definition and process Definition: The percentage of patients on Care Programme Approach who were followed up within 7 days after discharge from psychiatric in-patient care during the reporting period. Patients who are discharged from a mental health in-patient episode on a Care Programme Approach should receive a follow-up contact within seven days of the discharge. Relevant discharges include patients discharged to their place of residence, care home, residential accommodation, or to non-psychiatric care. All avenues must be exploited to ensure that the patients are followed up within seven days of discharge. Staff should be reminded to ensure accuracy of data written in patient notes. See R5. Date approved ready to be discharged entered on to Electronic Patient Record Follow up, by phone or face-toface, within 7 days? Yes Records should be checked to ensure type of meeting is valid to be called a 7 day follow up. See R1. No breach recorded Only patients who are discharged should be included within the data, not those on leave. See R2. No Has patient e.g.: transferred to inpatient? fall under CAMHS? died? Yes Clear documentation should be included in the patient notes and the HCP diary. See R4. Outside of reporting scope No Breach recorded Staff should have contact directly with the patient to count as a 7 day follow up. See R3. 6
Care programme approach 7 day follow up (continued) Approach We met with the Trust s leads to understand the process from discharge of a service user to the overall performance being included in the Quality Report. We selected a sample of 24 from 1 April 2014 to 31 March 2015 including in our sample service users who had and had not been followed up within 7 days. During our work we found four errors and therefore extended our sample by a further twelve. Of these further twelve samples we found three errors. The sample was not further extended due to the nature of the error. We agreed our sample of 36 to the underlying information held within the Rio System. Findings On two occasions the follow up date recorded was being pulled through incorrectly and showing the care coordinator attending the discharge meeting rather than the follow up meeting. This presents a data validity issue for the indicator as there may be other cases where the incorrect date is being pulled through and the follow up meeting does not take place within the 7 days, although within our testing the follow up meeting was found to take place within 7 days. Follow ups that are pulling through as taking place within zero days of discharge should be checked to ensure the date is valid and not pulling through the incorrect type of meeting. The parameters of the report should be amended to ensure the correct interactions are pulling through into the indicator. Recommendation 1. We are still to complete sample testing on day zero follow ups included in the calculation of the indicator to ensure valid 7 day follow ups did occur. One of the errors found related to where a patient was on leave but recorded as discharged. This individual was included in the indicator calculations in error, although the effect on the indicator was not material. It should be checked that cases are actually discharges and the patient is not on leave from the ward. Where a patient is on leave this should not be included within the 7 day follow up workings. Recommendation 2. A case was found where the 7 day follow up pulled through to the indicator was a call between the care coordinator and the crisis house staff team member. Per the Monitor guidance this does not qualify as a 7 day follow up as this should take place either face to face or by telephone with the patient. The testing showed that a 7 day follow up did take place within 7 days, therefore there was no effect on the indicator in this case. It is therefore recommended that the Trust should ensure that when a care coordinator speaks to the crisis house staff and not the patient themselves this is not counted as a 7 day follow up. Recommendation 3. 7
Care programme approach 7 day follow up (continued) Findings continued We identified cases where the notes documented within Rio had not followed the Trust s protocol for 72 hour follow up. These incidents related to interactions with the patient which met the definition of a follow up but were not documented within the notes as such. These meetings were followed by a formal 72 hour follow up as fully documented within the patient notes, but it is the earlier meeting that has been recorded within the indicator. If the Trust wishes to report on this earlier date the Rio progress notes and the HCP diary of the person carrying out the follow up should be completed to an appropriate standard to support this. The community team should be trained to complete the progress notes to the same standard as the care coordinator. It is recommended that clear documentation is recorded in the Rio notes as well as the HCP diary every time a meeting with a patient takes place. Recommendation 4. In 2 cases the discharge date was found to be different within the patient notes to the date per the report used to calculate the indicator. It could be seen that this was human error. Staff should ensure they check the date of entry and other items of this nature to help avoid this kind of discrepancy. In both these cases the 7 day follow up took place within the correct time frame. It is recommended that reminders should be sent to ensure care is taken by staff generally over information entered onto the system to ensure accuracy of discharges and follow ups. Recommendation 5. Our recommendation to introduce email reminders to complete the 7 day follow up was implemented during the year. During our discussions with staff this appears to be working well. Deloitte View: This is the second year that we have reviewed this indicator as part of our Quality Report external assurance. Of the six dimensions of data quality we found issues with accuracy, validity, reliability and relevance. Of these validity was found to be a significant issue. We have made five recommendations for improvement. 8
Access to crisis resolution home treatment team (gatekeeping) The Trust s process aims to perform a gatekeeping assessment on all patients of this type. Trust reported performance Target 2014/15 99.33% 95% Overall evaluation Indicator definition and process Definition: The proportion of inpatient admissions gatekept by the crisis resolution home treatment teams. Crisis Resolution / Home Treatment Services form part of the drive to ensure inpatient care is used appropriately and only when necessary, with service users being treated in the community setting, where possible. They are to provide a gateway to inpatient care and are deemed to have gatekept an admission if they have assessed the service user before admission and they were involved in the decision making process, which resulted in full admission. Patient requiring treatment A patient is assessed by the Crisis Team to see if they are suitable for admission or home treatment. All admissions are logged by the ward in a central admission Episode Dashboard Staff should ensure that they fill out all necessary documentation. See R6. The Crisis Team must complete the Referral Screening form (gatekeeping) Report Manager compares Referral Screening form with admissions Information regarding Gatekeeping performance is reviewed on a weekly basis by the Business and Performance manager. Investigate and document reason Yes Is there a fail? No No Breach recorded The performance is discussed and fed into the quarterly Board meeting and the Commissioner reports. 9
% of cases Access to crisis resolution home treatment team (gatekeeping) National context The chart below shows how the Trust compares to other organisations nationally for 2014/15, the latest national data available. The Trust is above the target but below the England average. Inpatient admissions with access to Crisis Resolution/Home Treatment teams - 2014-15 100% 98% 96% 94% 92% 90% Camden and Islington London providers Other English providers England average Target 2013-14 HTT gatekeeping rate Source: Deloitte analysis of Health and Social Care Information Centre data Approach We met with the Trust s leads to understand the process from identifying that a service user should have access to the crisis resolution team to the overall performance being included in the Quality Report. We selected a sample of 24 from 1 April 2014 to 31 March 2015 including both service users assessed by the Home Treatment Team and those who were not assessed. Of this sample all were found to have gone through the gatekeeping process and be recorded correctly. Findings The gatekeeping process was generally found to be working well. The Trust identified 6 occasions where gatekeeping did not take place in the year which were all found to be included within the indicator reporting. Our testing showed that whilst the gatekeeping process took place staff were not always efficient at filling out all of the details as required internally by the Trust such as ensuring patient notes state clearly that the gatekeeping process has been performed. It is therefore recommended that staff are reminded of all the information that is required from them when performing the gatekeeping process. Recommendation 6. Deloitte View: This was the first year that we have tested this indicator for the Quality Accounts external assurance. Overall we found that the data provided was of a good quality. Of the six dimensions of data quality we found issues with completeness. We have made a recommendation for improvement. 10
Local Indicator: Patient Safety incidents resulting in severe harm or death The Trust has improved its level of incident reporting and is now within the middle bracket of reporting nationally. Trust reported performance Target Overall evaluation 2014/15 2.57% No target set 2013/14 3.70% No target set 2012/13 2.90% No target set Indicator definition and process Definition: The Trust must report all patient safety incidents to the National Reporting and Learning System (NRLS). These incidents must be rated according to their degree of harm against the five categories: no harm, low, moderate, severe and death. This indicator is based on data submitted to the National Reporting and Learning System (NRLS), which was established by the National Patient Safety Agency (NPSA) in 2003 to enable healthcare staff to submit confidential reports to a national database about patient safety incidents (PSIs). PSIs are any unintended or unexpected incident, which could have, or did, lead to harm for one or more patients receiving NHS-funded healthcare. By submitting PSI reports to the NRLS, clinicians can then use the aggregate data collected to identify patterns in such incidents and opportunities to reduce risks to patients. All NHS healthcare providers are linked to the NRLS. All incidents are reported on the Datix system, using an online reporting form. All staff can access and have the ability to raise an incident report. The system is accessible from the intranet. The Trust has redesigned its website this year to make it easier for staff to log on to Datix and report incidents. Incident reporting forms may be submitted anonymously. The form includes details of the incident which occurred. Key fields are category, location, and degree of harm. Staff who detail the incident should provide as much information as possible at the outset. See R8. Patient Safety Incidents recorded by staff on Datix Incidents reviewed by Reviewer and Incident Reporting Officer Recommendation to provide more information to staff on how to classify severity of incidents. See R7. Incidents uploaded to NRLS Email notification sent to key staff inc. Matrons, Service Managers Serious Incidents escalated to Governance & Performance team and Executive team 11
Local Indicator: Patient Safety incidents resulting in severe harm or death (continued) National context According to the NSPA/NRLS published data on patient safety incidents reported by Trusts for the period 1 April to 30 September 2014, the Trust was within the middle of the reporting numbers of the mental health trusts in the country. This is the latest data available publicly. This is an improvement from the previous year when the Trust was one of the lowest reporters. This has been achieved by encouraging staff members to report all incidents even if they are not severe and ensuring the coding is correct for each incident. There has also been work around ensuring that all incidents are uploaded in a timely manner. The latest NSPA/NRLS data also shows that the Trust reports 50% of incidents within 7 days of them taking place, the national average is 26 days. It is widely recognised that organisations with higher levels of incident reporting are more likely to demonstrate other features of a stronger safety culture. Approach We met with the Trust leads to understand the process from reporting of incidents onto Datix, incident review process at the Trust, uploading of incidents to NRLS, and the overall performance being included in the Quality Report. We selected a sample of 24 from 1 April 2014 to 31 March 2015 in order to obtain a breadth of incidents across the period under review. We agreed our sample of 24 incidents to the underlying information held within Datix. Of the sample tested five errors were found. Given the nature of our findings we did not extend our sample. The local indicator is not subject to our limited assurance opinion. Findings Of the five errors, four relate to a difference in opinion of the severity of the incident. The incident is classified by the staff member who reports it but the severity reported depends upon their opinion. It is recommended that further details are provided for staff about how to classify incidents when they are recording them. A decision tree could be used to make the process more straightforward for staff members to classify incidents and ensure consistency. This is particularly the case around incidents that C&I could not stop happening. Recommendation 7. One of the errors found above was due to insufficient information being included by the staff member that reported the incident. It was therefore not possible to be able to substantiate the severity recorded. It is recommended that staff members who record incidents should provide as much information as possible initially. Recommendation 8. One of the errors found relates to where the same individual uploaded and checked the incident report. This was because the individual was the senior manager for the unit and there was no one more senior to check the notes. It is felt that this case was unique in its circumstances therefore no further recommendations are made around this. It was noted the all incidents tested were uploaded to NRLS, this is an improvement to the prior year. Improvements were seen this year in terms of the backlog for reporting all incidents and in a timely manner. This is reflected in our testing as well as the NRLS report. In addition to our recommendations, we have requested that the Trust amend the performance figure for this indicator within the Quality Report based on our recalculation of the performance for the year, which has been actioned by the Trust. Deloitte View: This is the third year that we have reviewed this indicator. Overall we found that the Trust has processes to generate a satisfactory quality of data underpinning the indicator. Of the six dimensions of quality we found some issues with one: accuracy, and have made some recommendations for improvement. 12
Recommendations 13
Recommendations for improvement Our recommendation to improve the quality of data Indicator Deloitte Recommendation Management Response Priority (H/M/L) CPA 7 Day Follow up 1. Correct follow up date recorded. Follow ups that are pulling through as taking place within zero days of discharge should be checked to ensure the date is valid and not pulling through the incorrect type of meeting. The parameters of the report should be amended to ensure the correct interactions are pulling through into the indicator. An automated report from the Trust Information Team is in place to support reporting on this metric. The report parameters are set to count whether or not a contract has taken place within 7 days of discharge (i.e. the contract outcome is recorded as patient seen ) for patients on CPA at the point of discharge. This is consistent with the Monitor definition. In the sample of 36 cases audited, there were 4 auditor queries. A review of these queries showed 2 cases in which the date provided in the automated report was the date of the CPA discharge review or a multi-disciplinary team care planning meeting. This included instances where a patient had returned from home to attend the formal discharge ward round. This could not be used as the 7 day follow up date in these two cases. H In order to provide further assurance regarding the data, a further validation check was undertaken. All contacts taking place within zero days of discharge have been manually checked to ensure contacts meet the appropriate definition. This data has been provided to the auditing team, and reflect the finding that in all but two cases, patients have been followed up within the 7 day period. To provide further assurance for future reporting, the Trust will now review all contacts within zero days manually to ensure data quality. Responsible Officer: Melissa Foster and Thomas Oscroft Timeline: 10/06/2015 Process for updating Council of Governors: Reported via the quarterly performance report. Full update given at scheduled Council of Governors meeting. 14
Recommendations for improvement Our recommendation to improve the quality of data Indicator Deloitte Recommendation Management Response Priority (H/M/L) CPA 7 Day Follow up 2. Genuinely a discharge It should be checked that cases are actually discharges and the patient is not on leave from the ward. Where a patient is on leave this should not be included within the 7 day follow up workings. The Trust performance team will work alongside the information team, aiming to introduce a function to the electronic system that reflects that a person is on leave rather than discharged from the inpatient bed. This will provide additional reporting assurance. Positively, it is evidenced in the re-audit undertaken that people are consistently followed up whilst on leave from the wards in at lease 48 hour, 72 hour and weekly intervals in addition to discharge follow ups. M The examples identified of patients being on leave rather than discharged have been exempted from the data. Responsible Officer: Melissa Foster and Thomas Oscroft Timeline: 10/06/2015 Process for updating Council of Governors: Reported via the quarterly performance report. Full update given at scheduled Council of Governors meeting. 3. Follow up with patient directly The Trust should ensure that when a care coordinator speaks to the crisis house staff and not the patient themselves this is not counted as a 7 day follow up as this does not meet the Monitor definition. This applies to two patients in the audit of 36 cases. These patients had been stepped down within the acute care pathway, and were in 24 hour care of psychiatric professionals. The follow up included a phone call to the 24 hour service, confirming the patient was settled and that there were no concerns. We have contacted the Monitor Policy Queries service to request clarity over whether this situation should lead to a positive count or to an exemption. M Responsible Officer: Acosia Nyanin and Melissa Foster Timeline: 31/05/2015 Process for updating Council of Governors: Reported via the quarterly performance report. Full update given at scheduled Council of Governors meeting. 15
Recommendations for improvement Our recommendation to improve the quality of data Indicator Deloitte Recommendation Management Response Priority (H/M/L) CPA 7 Day Follow up 4. Clear documentation Clear documentation should be recorded in the Rio notes as well as the HCP diary every time a meeting with a patient takes place. A Standard Operating Procedure is available for staff which makes clear RiO recording requirements. Requirements are also communicated to staff in the 72 hour follow up policy. It is positive to note that evidence shows that patients are consistently followed up, and that this is recorded in RiO, although the challenges around ensuring records are in both the HCP diary and the progress notes remain. Consistency of data entry can be a challenge, and we will recirculate this information to ensure all staff members are aware. M Responsible Officer: Melissa Foster Timeline: 31/05/2015 Process for updating Council of Governors: Reported via the quarterly performance report. Full update given at scheduled Council of Governors meeting. 5. Record entry Care should be taken generally over information entered onto the system to ensure accuracy of discharges and follow ups overall. The examples noted here were of discharge dates that were one to two days out, but still fell within the period of the audit. We have re-checked this data, and confirmed that this has not affected the reported percentages. From the Trust s perspective, this is a repeat of recommendation 4. Responsible Officer: Melissa Foster L Timeline: 31/05/2015 Process for updating Council of Governors: Reported via the quarterly performance report. Full update given at scheduled Council of Governors meeting. 16
Recommendations for improvement (continued) Indicator Deloitte Recommendation Management Response Priority (H/M/L) Gatekeeping 6. Processes fully completed Our testing showed that whilst the gatekeeping process always took place staff were not always efficient at filling out all of the details as required by the Trust such as ensuring patient notes state clearly that the gatekeeping process has been performed. It is therefore recommended that staff are reminded of all the information that is required from them when performing the gatekeeping process. The Trust acknowledge that there was one example in the sample of 24 cases where details were not recorded fully. We will reissue guidance to all appropriate staff to ensure that they are aware of the requirement. Responsible Officer: Aisling Clifford Timeline: 18/05/2015 Process for updating Council of Governors: Reported via the quarterly performance report. Full update given at scheduled Council of Governors meeting. M 17
Recommendations for improvement (continued) Indicator Deloitte Recommendation Management Response Priority (H/M/L) Patient safety incidents resulting in severe harm or death 7. How to rate incident severity. It is recommended that further details are provided for staff about how to classify incidents when they are recording them via Datix Web. A decision tree could be used to make the process more straightforward for staff members to classify incidents and ensure consistency. This is particularly the case around incidents that C&I could not stop happening. This recommendation relates to data uploaded to NRLS, and perhaps the understanding of this process has not been clarified. Trust staff do not upload incidents directly to NRLS or provide the classifications. Rather, all incidents are triaged by the Risk and Patient Safety team who report to NRLS as per national guidance. When someone is not an inpatient or present at one of the Trust community sites but comes to harm, they are nonetheless under our care and we are obliged to report this patient safety incident. The Trust has undertaken a significant piece of work over the last year to improve reporting, and are pleased to note from the latest NRLS data that we are now in the mid-range nationally for reporting of patient safety incidents (improved from being the lowest national reporters) and that we score the highest in London and joint highest nationally for timeliness of reporting. Responsible Officer: Margaret O Driscoll Timeline: 08/06/2015 M Process for updating Council of Governors: Reported via the quarterly performance report. Full update given at scheduled Council of Governors meeting. 8. Staff filling out full information Staff that record incidents should provide as much information as possible initially. Clear guidance is available for all staff members regarding incident reporting. As stated in the management response above, the initial report by staff members is an internal report, which is triaged by the Risk and Patient Safety team. Where consideration of further follow up is needed, a preliminary review is undertaken looking in more detail at the incident, at the response, and at immediate lessons to be learned. Dependent on the category of an incident and the level of harm, appropriate further investigation is undertaken in line with national guidance. L The Risk and Patient Safety Team will re-issue guidance currently available on the Trust Intranet as a reminder to all staff reporting via Datix Web (Trust Incident Reporting System). Responsible Officer: Margaret O Driscoll Timeline: 08/06/2015 Process for updating Council of Governors: Reported via the quarterly performance report. Full update given at scheduled Council of Governors meeting. 18
Update on prior year recommendations Our prior year recommendations have been mostly addressed but parts still require some work. Indicator Deloitte Recommendation Current year status CPA 7 Day Follow up Review clinical coding for discharge destinations The Trust should review the use of discharge destination coding and include the right coding destinations to enable correct calculations for the indicator. This should be followed by the issue of guidance to the information team and ward staff on the use and importance of entering the correct discharge coding for each patient. Responsible Officer: Melissa Foster Timeline: 18/07/2014 Improve case note documentation The Trust should review the quality of case notes to ensure that patient followup and the method of follow-up are captured accurately. Responsible Officer: Melissa Foster Timeline: 18/07/2014 Email alerts for potential delays to 7 day follow-up An automated email alert should be generated through the system once a delay is recorded on the Episode Dashboard. This email should be sent to the patient s Care Coordinator, Business Manager and the Team Manager to alert them to a potential delay to the 7-day follow-up. This email should be sent after the 3rd, 5th and 7th day of patient discharge depending on the follow-up status. Responsible Officer: Melissa Foster Timeline: 18/07/2014 It was noted that discharge destinations were reviewed and guidance has been completed and circulated. This has had some impact but it remains a challenge. This guidance will be reviewed in light of the new EPR systems go live in September 2015 to ensure further improvement. This recommendation was circulated to inpatient senior managers, ward staff and community senior managers and team manager to cascade to all care coordinators. The guidance also highlighted the importance of case note documentation. The Business and Performance Manager has a Dashboard to monitor all potential breaches and fails. This is reviewed every week. Incidences of where care coordinators or ward staff do not document the 7 day follow up plans in progress notes are followed up by email and they are reminded to record this. It was noted during the course of testing that the majority of case notes now clearly contain that a 7 day follow up is required, often the date that this is planned for is also stated. It was noted during testing that email alerts are in place and are emailed to care coordinators, Business Manager and Team Manager every day for 7 days or until a contact has been recorded on Rio. This has proved a useful and successful tool. 19
Update on prior year recommendations Our prior year recommendations have been addressed and this should be continued going forward. Indicator Deloitte Recommendation Current year status Patient safety incidents resulting in severe harm or death Patient safety incidents resulting in severe harm or death Improve reporting on NRLS, including coding of incidents and clear backlog of incidents. The Trust should investigate in detail why it is the lowest reporter of incidents on NRLS nationally. Responsible Officer: Acosia Nyanin Timeline: End of June, and then ongoing in terms of continuing with uploading and our checking system Continue to improve and strengthen staff communication around incident reporting. The Trust should continue with the staff communication campaigns around the quality and timeliness of incident reporting. Responsible Officer: Acosia Nyanin Timeline: Ongoing Extensive work has been undertaken internally and in liaison with the NRLS to improve the Trust s position. The Trust has identified and corrected coding errors where internal category codes were not mapped correctly to NRLS codes. Work has also continued to encourage and promote a good reporting culture within the Trust. The most recently published data from the NRLS (published 08/04/15) has demonstrated a 63% increase in the number of incidents reported to the NRLS between April 14 and September 14 when compared to the same period the previous year. The Trust is no longer one of the lowest 25% reporters nationally. The latest report published from the NRLS indicates the Trust occupies a position in the centre of the middle 50% reporters nationally. Work has continued to encourage and promote a good reporting culture within the Trust through weekly reminders to managers about reviewing and signing off incidents, monthly reviews of incident reporting at Divisional Performance Meetings as well as regular patient safety alerts on key areas of concern. The weekly monitoring report was introduced in September 14 and there has been a steady improvement in the number of incidents outstanding for review. This report is also reviewed at the monthly Divisional Performance Meetings and problem areas are highlighted for action. 20
Responsibility statement 21
Purpose of our report and responsibility statement Our report is designed to help you meet your governance duties What we report Our report is designed to help the Council of Governors, Audit Committee, and the Board discharge their governance duties. It also represents one way in which we fulfil our obligations under Monitor s Audit Code to report to the Governors and Board our findings and recommendations for improvement concerning the content of the Quality Report and the mandated indicators. Our report includes: Results of our work on the content and consistency of the Quality Report, our testing of performance indicators, and our observations on the quality of your Quality Report. Our views on the effectiveness of your system of internal control relevant to risks that may affect the tested indicators. Other insights we have identified from our work. What we don t report As you will be aware, our limited assurance procedures are not designed to identify all matters that may be relevant to the Council of Governors or the Board. Also, there will be further information you need to discharge your governance responsibilities, such as matters reported on by management or by other specialist advisers. Finally, the views on internal controls and business risk assessment in our final report should not be taken as comprehensive or as an opinion on effectiveness since they will be based solely on the procedures performed in performing testing of the selected performance indicators. Other relevant communications Our observations are developed in the context of our limited assurance procedures on the Quality Report and our related audit of the financial statements. We welcome the opportunity to discuss our report with you and receive your feedback. This report should be read alongside the audit plan circulated to you on 19 January 2015. Deloitte LLP Chartered Accountants 14 May 2015 This report is confidential and prepared solely for the purpose set out in our engagement letter and for the Board of Directors, as a body, and the Council of Governors, as a body, and we therefore accept responsibility to you alone for its contents. We accept no duty, responsibility or liability to any other parties, since this report has not been prepared, and is not intended, for any other purpose. Except where required by law or regulation, it should not be made available to any other parties without our prior written consent. You should not, without our prior written consent, refer to or use our name on this report for any other purpose, disclose them or refer to them in any prospectus or other document, or make them available or communicate them to any other party. We agree that a copy of our report may be provided to Monitor for their information in connection with this purpose, but as made clear in our engagement letter dated 19 th September 2012, only the basis that we accept no duty, liability or responsibility to Monitor in relation to our Deliverables. 22
Appendix 23
Data Quality Framework For evaluating the findings from our testing Overview The volume and importance of non-financial performance information across the NHS has grown significantly in recent years. Performance reporting has emerged as a key tool used both internally and externally. Managers use information to monitor performance, regulators use it to gauge risk, commissioners use it to ensure their priorities are met, and governors, patients and the public use it to gain more information about their trust and to hold them to account. Whilst the availability and use of non-financial performance information has developed quickly, the control frameworks used to produce and control such information has not been subject to the same level of rigour as that of financial information. On average a trust will receive information on 61 performance indicators on a monthly basis, but very few will be subject to independent review. This can result in a potential assurance gap. In the table below we have prepared a summary of key considerations that each trust should be able to answer regarding their performance information. It can be used as an assurance tool to gauge the risk around accuracy and completeness of performance information. Area Overview Key considerations System Governance The accuracy of an indicator is influenced by the level of automated vs manual controls. In general, an automated system requiring minimal manual adjustment has a lower risk of error. However, this assumes that the system controls are operating as they are intended. Accuracy and completeness of indicators are influenced by the tone at the top. Good performance would mean clarity of responsibility for performance metrics, clear processes and procedures in place for each metric which are regularly updated, and quick and comprehensive action where concerns have been raised. Is the indicator generated from one system or the interaction of different systems? How often are system controls reviewed to ensure they are appropriate and meet indicator definitions? How quickly is data produced after the event? Does data require manual adjustment prior to being reported as a performance indicator? Who is responsible for the quality and completeness of performance information at Board level? If different individuals are responsible for different indicators, is it clear who is responsible for each? Are there documented procedures and processes for each indicator and is this regularly updated? If data quality concerns have been raised have they been addressed quickly and comprehensively? Inputs Complexity and skill Some performance indicators rely on a wide variety of sources to produce the end metric. In general, the greater the number of separate sources of information, and the higher the volume of data, the greater the likelihood of error. Some indicators require specific skills to identify, analyse and report performance. Some indicators have complex rules, which require specialist consideration. If the complexity of these rules is not understood and applied correctly, there is a risk that indicators contain errors or are reporting incomplete information. What is the volume of inputs of each indicator on a daily / weekly / monthly basis? How many different sources of data are there, and how do you know they all apply consistent methodology in collecting and reporting the data? What checks are in place to ensure the consistency and completeness of input data? If performance indicators have specific rules, is there regular training to ensure that all individuals involved understand these rules and apply them correctly? Does the Trust have its own assurance systems in place to test compliance with such rules? Has the Trust got the appropriate skill and level of resources to identify, analyse and report performance for complex indicators? If national guidance is not clear, does the Trust have local guidance regarding process and procedures and is this shared with appropriate individuals? 24
Data Quality responsibilities The new False or Misleading Information offence applies to this year s Quality Accounts. New legal responsibilities over data quality From 1 April 2015, health providers are subject to the False or Misleading Information ( FOMI ) offence, introduced in response to issues over data quality in the NHS. The FOMI offence applies to: specified information which trusts already report regularly to the Health and Social Care Information Centre; and the contents of the Quality Accounts. The FOMI offence is a two stage offence: firstly, a NHS or private sector provider organisation is guilty of the offence if it provides information that is false or misleading whether intentionally or through negligence i.e. this is a strict liability offence where intent is not relevant to the offence being committed. secondly, if a provider has committed and offence, it is possible that a director or other senior manager or other individual playing such a role may be personally guilty of an equivalent of the FOMI offence as well. The potential penalties for providers include fines, a requirement to take specific action to remedy failures in data reporting, or to publicise that the offences have been committed and corrected data. For an individual, penalties can be an unlimited fine or up to 2 years in jail. Providers and individuals are able to make a defence that they reported information having taken took all reasonable steps and exercised all due diligence to prevent the provision of false or misleading information however it is currently unclear what would be interpreted as reasonable in this context. In practise, there is likely to be significant discretion exercised in determining whether to mount a prosecution. Deloitte view Over the course of the year, we have updated the Trust on the potential implications of the offence and have discussed with management the findings from our Quality Accounts work in the context of the offence. The scope of the FOMI offence is wide ranging, and covers many more indicators and data sets than are considered in our Quality Accounts data testing of three indicators, or than Internal Audit are able to cover in their data work each year. In order to be able to demonstrate across all reported metrics that they have taken all reasonable steps and exercised all due diligence to prevent the provision of false or misleading information, providers are ultimately reliant upon the quality of their systems for data recording and information reporting. However, accurately reported data is not just a compliance requirement it is perquisite for creating an insight driven organisation. A lack of accurate, complete and timely data can increase operational and financial risk. Failure to govern and use data effectively can lead to poor patient experiences and reputational damage. Data issues can also undermine a Trust s ability to run an efficient service, as key information that should influence decision making is not available or accurate. To support boards in considering their use of data, our latest NHS Briefing on Data Quality highlights areas of good practice for Trusts to consider in improving how they govern and use data. Key questions for Trust boards to consider include:: Is there a risk that your reported data is not accurate or that you are making decisions on unreliable data? What sources of assurance has the Board sought around the quality of data? Do you place too much reliance on the mandatory external data governance reviews to assure data quality? Is there an opportunity to improve patient outcomes, patient experience, operational efficiency and financial performance of your Trust by using data in a more sophisticated way? Has your Trust adequately identified the costs and benefits associated with a data governance effort? Does your Trust have in place a system of Data Governance designed to address data quality concerns and enable more effective data usage? Is your data governance effort owned at a sufficiently senior level and is the Board aware of data governance issues and concerns? Has your Trust set out its analytics and information vision and strategy? Is your analytics and information strategy aligned to other Trust strategies? Does your Trust have the analytics capacity, capability and technology to exploit its data assets effectively? 2015 Deloitte LLP. All rights reserved. 25
Other than as stated below, this document is confidential and prepared solely for your information and that of other beneficiaries of our advice listed in our engagement letter. Therefore you should not, refer to or use our name or this document for any other purpose, disclose them or refer to them in any prospectus or other document, or make them available or communicate them to any other party. If this document contains details of an arrangement that could result in a tax or National Insurance saving, no such conditions of confidentiality apply to the details of that arrangement (for example, for the purpose of discussion with tax authorities). In any event, no other party is entitled to rely on our document for any purpose whatsoever and thus we accept no liability to any other party who is shown or gains access to this document. Deloitte LLP is a limited liability partnership registered in England and Wales with registered number OC303675 and its registered office at 2 New Street Square, London EC4A 3BZ, United Kingdom. Deloitte LLP is the United Kingdom member firm of Deloitte Touche Tohmatsu Limited ( DTTL ), a UK private company limited by guarantee, whose member firms are legally separate and independent entities. Please see www.deloitte.co.uk/about for a detailed description of the legal structure of DTTL and its member firms.