Data Quality Review

Transcription

1 INFRASTRUCTURE, GOVERNMENT AND HEALTHCARE Data Quality Review London Borough of Hounslow 11 January 008 AUDIT

2 Content The contacts at KPMG in connection with this report are: Neil Thomas Engagement Director KPMG LLP (UK) Tel: Fax: Jennifer Dryden Senior Manager KPMG LLP (UK) Tel: Fax: Nick Rolfe Assistant Manager KPMG LLP (UK) Tel: Fax: Executive summary Stage one: Management Arrangements Stage two and three: Data Quality Spot Checks Appendices 1. Performance Improvement Observations. Follow up of Performance Improvement Observations. Summary of Scores 4. Best Value Performance Plan Report Page 4 5 This report is addressed to the London Borough of Hounslow and has been prepared for the sole use of the London Borough of Hounslow. We take no responsibility to any member of staff acting in their individual capacities, or to third parties. The Audit Commission has issued a document entitled Statement of Responsibilities of Auditors and Audited Bodies. This summarises where the responsibilities of auditors begin and end and what is expected from the audited body. We draw your attention to this document. External auditors do not act as a substitute for the audited body s own responsibility for putting in place proper arrangements to ensure that public business is conducted in accordance with the law and proper standards, and that public money is safeguarded and properly accounted for, and used economically, efficiently and effectively. If you have any concerns or are dissatisfied with any part of KPMG s work, in the first instance you should contact Neil Thomas, who is the engagement partner to the London Borough of Hounslow. Telephone , neil.thomas@kpmg.co.uk who will try to resolve your complaint. If you are dissatisfied with your response please contact Trevor Rees on , trevor.rees@kpmg.co.uk, who is the national contact partner for all of KPMG s work with the Audit Commission. After this, if you still dissatisfied with how your complaint has been handled you can access the Audit Commission s complaints procedure. Put your complaint in writing to the Complaints Investigation Officer, Audit Commission, 1st Floor, Millbank Tower, Millbank, London, SW1P 4HQ or by e mail to: complaints@audit-commission.gov.uk. Their telephone number is , textphone (minicom)

3 Section One Executive Summary The Audit Commission has developed a three-stage approach for assessing data quality, the first stage being a review of management arrangements for data quality, the second stage being the analytical review of those indicators flagged by the Audit Commission and the third stage being the completion of detailed spot checks of high risk indicators. The findings contributed to our conclusion under the Code of Audit Practice on the Authority's arrangements to secure value for money in relation to the specific criterion on data quality. We issued the following opinion as part of the conclusion on the 006/07 Audit. The Authority has a track record of using high quality information on costs to actively manage performance, improve value for money and target resources. Stage One The purpose of the work on management arrangements is to focus on the corporate data quality arrangements for the performance information prepared and used by the Authority. This is the second year in which we have undertaken work on data quality. In assessing the corporate management arrangements for data quality, we have looked beyond the previous focus on departmental systems and processes to consider how you are securing the quality of the data you use across the board. The review of management arrangements is structured around five themes: Governance and leadership People and skills Policies and procedures Data use Systems and processes These break down into thirteen Key Lines of Enquiry (KLoEs). We have assessed your arrangements against each KLoE and have scored you against each theme as defined below: Level Inadequate Adequate Performing well Performing strongly Description Below minimum requirements - inadequate performance Only at minimum requirements - adequate performance Consistently above minimum requirements - performing well Well above minimum requirements - performing strongly As a result of the work performed, we have assessed your overall performance as performing well. You have performed well in respect of your arrangements over policies and procedures and people and skills, however, further improvements are required in respect of your governance and leadership and systems and processes. We have found that the Authority has improved the score in five KLoEs as well as obtaining a performing strongly score in data use. There was also an improvement in an overall theme score (Governance and Leadership). However performance has declined in three KLoEs. Within KLoE 1.1 (Governance & Leadership), to increase the score going forward, goals should set at an individual level to ensure that data quality is produced in accordance with the data quality strategy with performance against this target monitored. For example each individual responsible for producing an indicator, could have a goal within their PDA to ensure that the performance indicator is produced accurately. Additional requirements have been added to KLoE. (Systems & Processes) in 007. There appears to be a fragmented approach to compilation of data at the Authority with a number of different systems used with varying strength of the control environment. The quality of performance information could be improved by the use of more controlled systems, with greater ability to record information, which can then be subject to interrogation and testing. In relation to KLoE 5. (Data Use), we have again identified weaknesses in the audit trails that have been maintained to support indicators. We have provided details of our key findings in section two of this report and have raised four performance improvement observations, summarised in appendix one. A summary of the scores for each theme and KLoE can be found in appendix three. Stage Two During stage two of the process we followed up issues arising from the analytical review of 006/07 Best Value Performance Indicators (BVPI) and non-bvpi data used in Comprehensive Performance Assessment (CPA) carried out by the Audit Commission. This analytical review informed our selection of a sample for testing at stage three. Two indicators were flagged up at this stage and included within our overall sample testing. These were:

4 Section One Executive Summary BV184a BV165 Decent homes found to be fairly stated due to real performance improvement; and Percentage of pedestrian crossings with facilities for disabled people found to be fairly stated as a real decline in performance and tested at stage three also (see below). Stage Three When choosing how many and which Performance Indicators (PIs) to review at stage three, in addition to those identified for review by the Audit Commission, we used the results from stage one and our cumulative audit knowledge and experience to determine the total number of PIs for review. From the prior year data quality review, we found that arrangements for ensuring data quality were largely adequate, however raised two reservations on the library performance indicators. These reservations were subsequently revised due to the fact problems were identified nationally throughout library data quality. We therefore decided to follow up these PIs in our spot check this year and the results of testing for this year is shown below. We identified three BVPIs and non-bvpis where we have chosen to perform a review: BV165 : Percentage of pedestrian crossings with facilities for disabled people figure was amended and was fairly stated; BV14: Repeat homelessness figure was amended and was fairly stated; and Public Library Service (PLS7):Assessment of users 16 and over of their library service figure was fairly stated. For stage three, in addition to the specified indicators detailed above, we selected the two Institute of Pubic Finance Performance (IPF) indicators. These were: library stock turn and stock level. In the previous year, these indicators has been subject to a reservation due to the lack of audit trail in place. We were able to test these indicators this year, although the report for stock level had not been run on the correct date. Testing found the indicator to be fairly stated. The results of these spot check reviews indicate that the data quality underpinning your PIs was found to be largely adequate. No reservations were issued from this year s review, however we did identify amendments to BV165 and BV14. The results of our detailed spot check of performance indicators are summarised in section three. Best Value Performance Plan Report As in prior years, we have also audited your Best Value Performance Plan 007/08 in accordance with section 7 of the Local Government Act 1999 and the Audit Commission s Code of Audit Practice. The opinion in relation to this was issues on the 0 December ahead of the national deadline of the 1 December 007. A copy of our report is found at Appendix four. Performance Indicators: Interim Review In January 007 we undertook a review of the systems and processes supporting the production of your PIs. The review included the testing of a sample of high risk PIs, as prescribed by the Audit Commission. We reported the findings of this review to the Authority in May 007. Our work found that whilst there were areas for development in the control environment surrounding the production of a number of the PIs, the majority of these development opportunities could be addressed by improving the quality of the review of calculations and the detail of audit trails recorded. We raised 11 indicators as part of that review.

5 Section Two Stage One: Management Arrangements As a result of the work performed, we have assessed your overall level of performance as performing well. You have performed well in respect of your arrangements over policies and people and skills, however, further improvements are required in respect of your Data Use (KLoE 5. see appendix three), Governance and Leadership (KLoE 1.1) and Systems and Processes (KLoE.). The table below sets out details of areas where you are currently meeting requirements and where further development is required: Theme Governance & Leadership Policies & Procedures Systems & Processes People & Skills Data Use Score Performing well Performing well Performing well Performing well Performing well Summary of key issues The key drivers behind this score are linked to the development of a corporate data quality strategy, embedding a framework for monitoring data quality and communication of your commitment to data quality to all staff responsible for the preparation of performance information. The Authority have worked on developing the data quality strategy and this was published in May 007. Accountability for data quality has clearly been set out in the appendix to this. An objective for data quality needs to be incorporated in the corporate plan to emphasise the importance of data quality throughout the Authority. (see Performance Improvement Observation (PIO) One in Appendix One). The key drivers behind this score are linked to the development of a data quality policy supported by a set of operational procedures and guidance which are monitored and reported to senior management. There are policies and procedures in place to secure data quality at the Authority. This has been enhanced by the introduction of the data quality strategy in May 007. To reach the next level, the Authority should expand the application of the data quality policy to Partner organisations. This would ensure that data received from partnerships and used in PI production, has been verified and checked to the same standards as internally produced data. The key drivers behind this score are linked to the strengthening of your performance information system controls and performance of detailed scenario planning for your performance information systems. The Authority has improved systems and processes to ensure quality of data. There is a Performance and Management Group incorporating a member of the Executive and senior managers from each directorate to discuss data quality. The Authority should ensure reviews are performed on data used for outturn figures by other members of staff, for example the PI Co-ordinators. This should ensure fewer amendments and allow the Authority to develop and improve, see PIO. Good practice was noted in the housing department whereby reviews were carried out and errors found and corrected prior to our audit. This KLoE considers the roles and responsibilities in relation to data quality, assessing whether they are clearly defined or documented. Ongoing training on data quality is available. This is provided to those individual who have a specific training need, as identified by management. Staff are invited to attend a workshop provided by Policy and Performance Unit. External Audit also presented at the workshop to outline our requirements of the audit. Job descriptions have been developed to include data quality and set their responsibilities. In order to develop further, the Authority should ensure data quality objectives are set for officers that have responsibility for data quality and the compilation of performance indicators. (see Performance Improvement Observation (PIO) Four in Appendix One). You have put in place arrangements that are focused on ensuring data supporting performance information is used to manage and improve your delivery of services, and you have effective controls in place for data reporting. The use of data to support service improvement has been evident this year. A number of indicators have identified where services could be improved and resources have been allocated as such. However the Authority should ensure there are clear audit trails to support data; in particular the library indicators where reports were not run at the correct date and reports could not be run retrospectively (see PIO two). In order to improve and develop, the Authority should ensure that reports are maintained to support all data used to compile figures and those reports are run on the correct dates. 4

6 Section Three Stages Two and Three: Data Quality Spot Checks Our analytical review work at stage two identified that the PI values reviewed fell within expected ranges and were substantiated by evidence. We carried out spot checks as per Audit Commission guidance on two of your performance indicators. As a result of our audit work no PIs were amended and no reservations issued on the PIs as summarised in the table below. PI Description Original Value Value stated Conclusion Stage two BV184a Decent Homes 0% 0% Figure was found to be fairly stated. The Authority completed its decent homes standard, making all homes decent apart from a small number where access to properties was denied by residents. However the result of this was less than 0.5% and therefore was rounded to 0%. BV165 Percentage of pedestrian crossings with facilities for disabled people 41.86% 41.8% Figure was amended and is now fairly stated. The use of the team from Camden Consultancy Service to carry out the survey of crossing compliance, led to a significant reduction in performance as crossings were found not to comply with the definition. Stage three BV165 Percentage of pedestrian crossings with facilities for disabled people 41.86% 41.8% Figure was amended and is now fairly stated. As above, our spot found one error but this was found to be one off where one of the crossings did not have all the necessary facilities. BV14 Repeat homelessness 1.7% 1.1% Figure was amended. Duplicate cases with multiple decisions were found, meaning the figure was overstated and Council s performance therefore improved. IPF Stock Turn: Books issued / books available for issue per 1,000 population Figure was fairly stated. However the stock level figure was found to have been run on the incorrect date. Testing of this figure showed this did not have a significant impact on the outturn figure and was within acceptable limits. IPF Stock Level per 1,000 population 1,56 1,56 Figure was fairly stated. As above problems noted with audit trail. PLSS7 Percentage of users aged over 16 who view their library service as very good 91% 91% Figure was found to be fairly stated. Results from the CIPFA survey could be agreed to the outturn figure. 5

7 Appendix 1: Performance Improvement Observations This appendix summarises the performance improvements that we have identified relating to the KLoE criteria while undertaking our assessment of your data quality management arrangements. We have given each of our observations a risk rating (as explained below) and agreed with management what action you will need to take. We will follow up these performance improvement observations as part of our interim audit. Priority one: We believe that these actions require immediate attention by the Council to remedy or manage issues that currently present a significant business risk. Priority rating for performance improvement observations raised Priority two: We believe that these actions requires short to medium term action by the Council to remedy or manage issues that currently present some business risk Priority three: We have identified these as an area where we believe that the Council might consider implementing known best practice, they do not necessarily represent an area where there is currently weakness. No. 1 Priority Performance improvement observation Governance and leadership: Data quality and corporate objectives There is no specific objective set in the corporate plan with regards to data quality. There is reference to improving the CPA assessment, but this does not incorporate improving the quality of data that is used to produce performance indicators. We recommend that an objective is incorporated to the corporate plan to show the importance of ensuring data is accurate, can be supported and is reviewed on a regular basis. Data Use: Maintenance of an Audit Trail There have been improvements made to the audit trail in 07/08. However we still have had difficulties in obtaining a clear audit trail for the library indicators and BV165. For the library indicators, the stock report had been run after the year end meaning we did not have a stock figure as at 1 March 007. We did manage to gain assurance over this figure by considering additions and deletions for the period. For BV165, there were delays in providing evidence for TfL maintained crossings in the Borough. We have downgraded this to level this year as there have been improvements made. We recommend the Authority maintains clear audit trails for all figures used, and reports are extracted on the correct dates as per the guidance. Systems and processes: Review We have noted that in some cases, there is no review of figures prior to submission for audit. We identified a number of errors in the calculation of figures, that would likely be picked up with more strenuous review. We recommend that a review of all figures is carried out, if possible by an independent member of staff to ensure greater accuracy in calculation. We noted good performance of this in the housing department, where amendments were notified to us prior to us beginning the audit through vigorous review of data. This should be encouraged throughout the Authority. Management response An objective in regards to data quality will be incorporated to the Hounslow Plan, which is currently being rewritten. PMG & PI Co-ordinators have been reminded of the importance of ensuring that all data produced is audit compliant. This includes maintaining clear audit trails, reviewing calculations and extracting reports on the correct dates for all performance indicators. This message is also communicated in the DQ Strategy, and at the DQ workshops. PMG & PI Co-ordinators have been reminded of the importance of ensuring all data produced is audit compliant. This includes maintaining clear audit trails, reviewing calculations and extracting reports on the correct dates for all performance indicators. Officer and due date Policy & Performance Unit. (Chris Bates) Performance Management Group Performance Management Group This message is also communicated in the DQ Strategy, and at the DQ workshops. 6

8 Appendix 1: Performance Improvement Observations No. 4 Priority Performance improvement observation Governance and leadership: Member training Member training should be provided which outlines the Councils approach to data quality, together with the approach adopted in order to address specific risks identified. Management response This recommendation will be factored in the Members Training programme. Officer and due date Policy & Performance Unit & Organisational Development 5 People and skills: Objective setting We found that objectives were not being included in individual officers appraisals. We recommend objectives in relation to data quality should be included within personal goals and assessed as part of the annual appraisal. Where objectives are set, these should be SMART to ensure that they are clearly demonstrable. For example an officer could have the objective of ensuring their PI is calculated accurately in accordance with the guidance and clear audit trails are maintained. This recommendation will be incorporated in the PDA Guidance for Managers. This guidance will be available to all staff via the Intranet by February 008. Policy & Performance Unit and Organisational Development 7

9 Appendix : Follow up of Performance Improvement Observations This appendix summarises progress against those performance improvement observations which we raised as part of our interim review, together with those performance improvement observations raised as part of our prior years review. We outline those areas below where further action is still required. The risk grading is aligned to that detailed in Appendix One. # Risk (one) Performance Improvement Observation Recommendations raised as part of the interim review of PIs (January 007) 1 Maintaining an adequate audit trail to (one) support the figures included in the indicator Our audit found that in the following cases a full audit trail did not exist because information from live systems was not recorded at the time of calculation: BV 1 BV 1b L71 a & b LAA 7 You should ensure that when an indicator is reliant on data from live systems, the relevant reports supporting the figures in the calculation should be printed off or saved at the time of the calculation in order to provide a clear audit trail. Conducting formal reviews of calculation by a separate individual For all the indicators tested we found that a review process existed. However, for the following indicators we found that this review was not sufficient to detect basic errors: BV 15b BV 1 LAA 8a LAA 4a Sufficient detail on the calculation of the outturn figure should be provided to the reviewer to allow them to re-perform the calculation. The reviewer should then review the calculation in sufficient detail to ensure no data extraction or calculation errors have occurred. Management response BV 1: We will ensure the relevant reports are captured at the end of year. BV 1b: The raw data is still held at local level - it is copies of this information that is sent to the centre for the indicator to be calculated. We are putting measures in place to ensure that this does not reoccur. L71a&b: Officers have been advised and have implemented processes to save a copy of this spreadsheet when quarterly figures are generated in order to provide an audit trail. LAA 7: The data provided in each quarter with backing documentation is going to be different from the figures provided by the Police at the end of the financial year. We will at the end of the financial year reconcile all figures and there will be a final calculation, which will represent our end of year performance. BV 15b: The outturn calculation will be verified by the PI co-ordinator prior to submission and will also be subject to sign-off by the relevant Service Heads and SMT. LAA 8a: The ASB Policy Officer is now including file numbers in the returns submitted to the Community Safety Team and this data is being verified by another officer within the team. Update From our data quality review we found that all reports had been run to generate figures for the PIs. However for the libraries indicators, we have identified that the stock level report was not run on the required date and there is no ability to retrospectively run the report as it is a live system. Although this did not mean the PI was materially misstated, best practice would mean staff should ensure reports are run on the correct date. We have found that some PIs are still not subject to appropriate review. This led to some basic errors being made in calculation. See PIO in appendix 1. However we have found performance is improving, in particular in the Housing Department where extensive review was made and adjustments identified prior to our audit. 8

10 Appendix : Follow up of Performance Improvement Observations # Risk (one) (one) (one) Performance Improvement Observation IPF Library indicators stock turn and level At the time of our audit no progress had been made on the library indicators since our Data Quality audit last year. However, after discussing the issue with CIP they were able to produce a report which could identify all issues by book reference number. To ensure we can complete our Data Quality testing this year, a similar report needs to be created for books in stock. BV 15a & b Street lighting Our testing of these indicators identified that the CONFIRM system should be prevented from defaulting the reporting/detection date to the current date to help avoid data entry errors. HIP BPSA - Percentage of planned to responsive repairs Our testing of the outturn figure identified a number of issues which need to be addressed before the outturn figure is calculated: Void repairs should been included as responsive works. Some repairs are coded as EM, PV and UR; the nature of these repairs should be determined so they can be correctly classified. Jobs to be completed in 40 days should be classified on whether they are planned or responsive rather than value. Voids are should be recorded completely on OHMS; therefore, so total expenditure can be apportioned correctly. This issue has been fixed since December 006, but needs to be retrospectively reviewed and corrected from April to November 006. Expenditure on hostels and sheltered accommodation should be included in the calculation. Improving your Data Quality Strategy (1) Our benchmarking exercise found the following examples of improved practice you should incorporate within your Data Quality Strategy: for each system you should keep procedure notes which highlight areas of best practice approach; creating list a Frequently asked questions on data quality with answers, available to all your officers on the intranet. Management response KPMG, CiP and DS Ltd have been in contact to resolv e these issues with the libraries system. Reports have now been produced and KPMG will be on site early July to check the system as part of the Main Data quality Review 006/07 We are also reviewing our reporting/recording procedures to ensure that all jobs are correctly recorded in future to prevent such issues arising again. The issues raised by KPMG have therefore been corrected and processes amended where required We have amended our processes which have altered performance considerably, and will stick with this to ensure we are measuring correctly for now and the future. We have identified one description. The other two codes were entered in error. We will ensure that data is checked for errors before running. In future jobs will be classified on the basis of the indicator definition and not on the basis of value. We have also identified why the ledger and OHMS do not reconcile and this is an issue with management and other administrative costs which are applied at year end to the ledger, and therefore do not show in the OHMS report. We have now included Hostel and Sheltered accommodation repairs into performance, which are now easily identifiable also. This has now been included in the DQ Strategy PPU are currently in the process of setting up, and making available, a Frequently asked questions link on the Intranet Update We have obtained the necessary reports in order to complete the audit. Although one was not run on the correct date. Issue has been corrected as reviews have been carried out regularly to ensure jobs are recorded correctly. Reclassifications have been made and other issues have been taken account of in the calculation of the Performance Indicator. Included in DQ Strategy. 9

11 Appendix : Follow up of Performance Improvement Observations # Risk (three) (three) (three) (three) Performance Improvement Observation Improving your Data Quality Strategy () Our benchmarking exercise found the following examples of best practice, which you could aim to incorporate within your Data Quality Strategy as it becomes embedded within your processes: you should ensure full audit trails for data collection and processing are captured and interrogated, so errors can be traced back to individual users. an e-learning package on data quality should be developed and made available to all staff. LAA 4a Number of four week quitters attending NHS Stop Smoking Services The outturn figure should be recalculated based on: the new two week quit period; the higher threshold of 10ppm for CO breath content. KPI Users who have moved on in a planned way from temporary living arrangements The source for the figures used to compile this indicator are the quarterly performance returns supplied to the authority by the providers. It is recommended that these figures are reviewed by a member of LBH external to the Supporting People team, as an added control. It is recommended that the Supporting People Manager and the Performance Officer should carry out regular reconciliations of the data held on SWIFT and the data held on the Performance Officer s spreadsheets in order to identify any discrepancies. BV b % childcare settings that have input from qualified staff In two of the ten cases sample tested, it was found that the settings included in the indicator as having qualified staff had no such staff. Although childcare settings are supposed to inform you when teachers move this does not always happen. Therefore, some changes are only identified on an annual basis during your staff audit. Each childcare setting should complete a quarterly return on the number of staff with qualifications, which is then used to compile the outturn figure. Management response Systems such as SWIFT, Agresso, iworld, and OHMS have log in records, which allow errors to be traced back to individual users. E-learning is possibly something the Council will look to progress in the future. The two patient files that were missing when audit occurred have been identified and now agrees with figures reported. It was confirmed with Department of Health that local measurement was in line with their guidance on how to measure four week quitters. We agree with the comments made regarding this performance indicator. Supporting People had identified the improvement themselves and will be working with West London to look at the issue of collectively auditing providers. The yearly staff audit of qualifications (BVa) will now be recorded on EMS Early Years Module and Training Manager Module. Work has just been completed on populating data into EMS (Education Management System) to ensure that a more robust system is in place that can provide date stamped information and retrospective information on staff that had been employed at a particular point in time and the qualifications held. This information will be updated each term and as and when settings advise of any changes outside the specified collection schedule. Settings will be asked to update their staff details by checking the details we hold on file via a print out of EMS, which will be sent out each term ( times a year). This will compliment the annual staff qualification audit, which takes place each January. Update No e-learning implemented at the moment. Errors can be traced. As per management response. Supporting People identified the improvement themselves and have worked with West London to look at the issue of collectively auditing providers. The process around recording the qualifications of staff in childcare settings are now more robust and have improved significantly. 10

12 Appendix : Follow up of Performance Improvement Observations # 11 Risk (three) Review of data quality 1 (one) (one) Performance Improvement Observation LAA 10 no. of year olds placed and retained for first year of course The definition should be tightened to specify the relevant timeframe for each reporting period. This should be either: the Quarter x figure should be the number of who have been placed on a course 1 months previously who are still attending; or the Quarter x figure should be the number of year olds who were placed on a course in Quarter x and attended the course for 1 months. the first definition would be more preferable given the long time lags in reporting based on the second. Policies and procedures: Development of a Data Quality Policy A standalone data quality policy should be developed and distributed alongside the practical guidance. This should incorporate processes for data collection, recording, analysing and reporting of data quality and be implemented in all business areas. The data quality policy should reinforce the first time right philosophy. This should include arrangements that are in place with partnership bodies thereby ensuring that data received via rd meets the same data quality requirements as internally generated data. Data Use: Maintenance of an Audit Trail The audit resulted in the provision of two reservations on the library indicators. This reflected the fact the libraries system has not captured data which would allow us to agree the data on which the calculation is based back to source. The Council should review the system capabilities in order to assess as to whether the level of data captured can be expanded such that audit verification is possible. Management response I am confident we will have the data we need from LSC in time for the Q4 monitoring. The Council will be developing a DQ strategy in the coming months to support the Business Planning & Performance Management Guidance currently used by all officers involved in business planning and PI s. The strategy will cover all the recommended points as outlined in this report. The Council will be formally responding, to the Commission highlighting our concerns on the late notification of the audit requirements. CIP will be working closely with the Galaxy system provider to make the necessary systems modifications to capture all relevant data. Update Relevant information received from the LSC. DQ strategy has been developed and distributed throughout the Council. Improvements have been made to the reporting available. We have been able to audit the PIs in 06/07. However the report was run nearly a month late but this did not affect the outturn figure to make it materially misstated. 11

13 Appendix : Follow up of Performance Improvement Observations # 4 5 Risk (one) Performance Improvement Observation Review of data quality (cont) Systems and processes: Systems mapping To support the development of the data quality strategy, the Council should undertake a data mapping in order to determine the key systems contributing data. This process should include data which is generated manually. Following the mapping exercise, the Council should undertake a risk assessment of each system. This will ensure that actions taken to address any weaknesses identified are cascade on a risk basis. Risks should be reflected within the risk register. As part of this process, business continuity arrangements in place for each system should be considered. Governance and leadership: Development of action plans In support of your data quality policy, the Council should develop a standalone delivery and action plan identify key milestones for improvement which are monitored and reported to CMT. Governance and leadership: Member training Member training should be provided which outlines the Councils approach to data quality, together with the approach adopted in order to address specific risks identified. Management response The Council will be undertaking a Systems mapping exercise following the agreement of the DQ strategy. The Mapping exercise will cover all systems used to collect BVPI s. A risk assessment will form part of the action plan. A stand alone delivery and action plan outlining key milestones for improvements will follow on completion of priority number 1 &. Members training will be provided. Training will include outlining the Councils approach to data quality in order to address specific risks identified. Update The mapping exercise has been undertaken and risks have been assessed. Our review at Stage 1 in 06/07 has incorporated this. Milestones and action plans have been identified in the Data Quality strategy. There has not been any member training carried out to date on data quality. 6 People and skills: Objective setting Objectives in relation to data quality should be included within personal objectives and assessed as part of the year end process. Where objectives are set, these should be SMART to ensure that they are clearly demonstrable. This recommendation will be covered within the DQ strategy and DQ Action Plan. From discussion with various staff members, data quality objectives have still not been routinely set as part of the appraisal process. 7 (three) Governance and leadership: Risk Management Where data quality risks are identified, for example as a result of data cleansing exercises, these should be linked to and reflected within the risk register. Where risks are identified these will be reflected within the Risk register. This recommendation will be covered within the DQ strategy. No risks identified in the risk register in relation to data quality but this should continue to be assessed and reviewed. 1

14 Appendix : Summary of Scores Theme/Key Lines of Enquiry (KLoE) Governance and leadership 1.1 Responsibility for data quality is clearly defined 1. You have clear data quality objectives and these are formally documented. 1. You have effective arrangements for monitoring and review of data quality. Policies and procedures.1 Organisational policy for data quality has been defined and is in place, supported by a current set of operational procedures and guidance. Policies and procedures are followed by staff and applied consistently throughout the organisation. Systems and processes.1 You have appropriate systems in place for the collection, recording, analysis and reporting of the data used to monitor performance, and staff are supported in their use of these systems.. You have appropriate controls in place to ensure that information systems secure the quality of data used to report on performance and to keep top management aware of necessary action in relation to data quality... Security arrangements for performance information systems are robust, and business continuity plans are in place..4 Standards are specified for shared data or data supplied by third parties. People and skills 4.1 You have communicated clearly the responsibilities of staff, where applicable, for achieving data quality. 4. You have arrangements in place to ensure that staff with data quality responsibility have the necessary skills Data use 5.1 You have put in place arrangements that are focused on ensuring that data supporting performance information is also used to manage and improve the delivery of services. 5. The council has effective validation procedures in place to ensure the accuracy of data used in reported performance indicators As can be seen from the table above, performance has improved in five KLoEs and declined in three. The reasons for the decline in these are shown below: 1) KLoE 1.1 We could not obtain evidence that goals were being set for individuals at the Authority around ensuring data produced is of a sufficient quality. For example a goal around ensuring that the performance indicator they are responsible for is produced accurately could be used in their PDA. This can be measured by the outcome of an audit, either externally or internally. See PIO 4 in appendix one. ) KLoE. Different requirements of the KLoE this year. More focus on the use of information systems to record and report information. There appears to be a fragmented approach to compilation of data at the Authority with a number of different systems used. ) KLoE 5. Our audit last year identified significant weaknesses in audit trails leading to reservations of two indicators and a reduced score this year at Stage 1. Improvements have been made but some weaknesses have still been identified. See PIO in appendix one. 1

15 Appendix 4: Best Value Performance Plan Report Auditor s Report to the London Borough of Hounslow on its Performance and Improvement Plan for the Financial Year 007/08 Certificate We certify that we have audited the London Borough of Hounslow best value performance plan in accordance with section 7 of the Local Government Act 1999 ( the Act ) and the Audit Commission's Code of Audit Practice ( the Code ). This report is made solely to the Council, in accordance with Section 7 of the Act. A copy of this report will be sent to the Audit Commission under section 7(5)(b) of the Act in relation to our recommendation to the Audit Commission under section 7(4)(e). A copy of this report will be sent to the Secretary of State under section 7(5)(c) of the Act if we include a recommendation under section 7(4)(f) that the Secretary of State should give a direction under section 15 of the Act. Our audit work has been undertaken so that we might state to the Council, to the Audit Commission and (where necessary) to the Secretary of State those matters we are required to state to them in such an auditor s report and for no other purpose. To the fullest extent permitted by law, we do not accept or assume responsibility to anyone other than (i) the Council, for our audit work, for this report, or for the opinions we have formed, (ii) the Audit Commission, for our recommendation under section 7(4)(e) and (iii) the Secretary of State, for our recommendation (if positive) under section 7(4)(f) of the Act. Respective Responsibilities of the Council and the Auditor Under the Local Government Act 1999 (the Act) the Council is required to prepare and publish a best value performance plan summarising the Council s assessments of its performance and position in relation to its statutory duty to make arrangements to secure continuous improvement to the way in which its functions are exercised, having regard to a combination of economy, efficiency and effectiveness. The Council is responsible for the preparation of the plan and for the information and assessments set out within it. The Council is also responsible for establishing appropriate performance management and internal control systems from which the information and assessments in its plan are derived. The form and content of the best value performance plan are prescribed in section 6 of the Act and statutory guidance issued by the Government. As the Council s auditor, we are required under section 7 of the Act to carry out an audit of the best value performance plan, to certify that we have done so, and: to report whether we believe that the plan has been prepared and published in accordance with statutory requirements set out in section 6 of the Act and statutory guidance and, where appropriate, recommending how the plan should be amended so as to accord with statutory requirements; to recommend: where appropriate, procedures to be followed in relation to the plan; whether the Audit Commission should carry out a best value inspection of the Council under section 10 of the Local Government Act 1999; whether the Secretary of State should give a direction under section 15 of the Local Government Act Opinion For the purpose of forming our opinion whether the plan was prepared and published in accordance with the legislation and with regard to statutory guidance, we conducted our audit in accordance with the Code. In carrying out our audit work, we also had regard to supplementary guidance issued by the Audit Commission. We planned and performed our work so as to obtain all the information and explanations, which we considered necessary in order to provide an opinion on whether the plan has been prepared and published in accordance with statutory requirements. In giving our opinion we are not required to form a view on the completeness or accuracy of the information or the realism and achievability of the assessments published by the Council. Our work therefore comprised a review and assessment of the plan and, where appropriate, examination on a test basis of relevant evidence, sufficient to satisfy ourselves that the plan includes those matters prescribed in legislation and statutory guidance and that the arrangements for publishing the plan complied with the requirements of the legislation and statutory guidance. Opinion In our opinion, the London Borough of Hounslow has prepared and published its best value performance plan in all significant respects in accordance with section 6 of the Local Government Act 1999 and statutory guidance issued by the Government. KPMG LLP Chartered Accountants London 0 December