Item 10 Appendix 1d Final Internal Audit Report Performance Management Greater London Authority April 2010

Transcription

1 Item 10 Appendix 1d Final Internal Audit Report Performance Management Greater London Authority April 2010 This report has been prepared on the basis of the limitations set out on page 16.

2 Contents Page Executive Summary 3 Observations and Recommendations 9 Appendix 1 Audit Framework 14 Appendix 2 - Staff Interviewed 15 Statement of Responsibility 16 This report and the work connected therewith are subject to the Terms and Conditions of the contract dated 20/06/2007 between Greater London Authority and Deloitte & Touche Public Sector Internal Audit Limited. The report is produced solely for the use of Greater London Authority. Its contents should not be quoted or referred to in whole or in part without our prior written consent except as required by law. Deloitte & Touche Public Sector Internal Audit Limited will accept no responsibility to any third party, as the report has not been prepared, and is not intended for any other purpose.

3 Executive Summary Introduction and Background 1. This audit forms part of the 2009/10 Internal Audit Plan, which has been approved by the Mayor and the Audit Panel. The plan entails a review of the systems and controls operating over the Authority s Performance Management Framework. 2. A new Performance Management Framework was implemented for the 2009/10 financial year in line with the Authority s Strategic Plan The Framework is the responsibility of the Chief Executive and managed by the Strategic Management and Delivery Unit (SMDU). 3. In order to manage and review the performance of the Authority's strategic plan, performance management focuses on six outcomes: Directorate strategies, plans and programmes and projects; High level targets (with clearly defined deliverables and where possible, numerical performance indicators); Chief Executive Delivery Meetings (CEDM) (to discuss progress against targets and outcomes for each directorate). The output of this are notes to confirm the actions to be taken, with agreed timescales for implementation; Priority reviews (meetings to discuss specific strategies/projects in more detail); Exception meetings (if required, based on delivery meetings with the relevant Assistant Director / Head of Unit and Director); and Quarterly Performance Reports, summarising overall progress and performance. 4. 'The GLA's Performance Management Framework aims to ensure that the whole organisation is unified in its approach, delivering the same aims and objectives, securing more efficient and effective delivery' (Strategic Plan ). 5. Evidence was gathered through discussion with relevant staff members and samples of prime source documents were selected and tested to evaluate the effectiveness of the controls in operation. A summary of the findings is contained within the following paragraphs. Performance Management Framework 6. The Performance Management (PM) Framework has been clearly documented. The document set outs: the structure of the framework; the processes in place to make the framework work; how the framework applies to different levels of the organisation; the team/unit, individual and organisational arrangements; the performance monitoring and management cycle for the financial year; roles and responsibilities at the different levels of governance; and a diagram illustrating the flow of information. Item 10 Appendix 1d Page 3

4 7. The Performance Management Framework was formally signed off on 9 th February 2010 by the Budget Monitoring Sub-Committee. 8. Guidance on the Performance Management Framework is available on the Strategic Management and Delivery (SMD) home page on the Intranet. 9. The SMD pages also provide more detailed information on the quarterly monitoring arrangements and other aspects of performance management. This includes the Chief Executive s Delivery Meetings with links to the terms of reference (TOR) and resulting actions log, the rationale and handbook for corporate health indicators, the strategic plan monitoring arrangements and the strategic plan guidance, including guidance on the Strategic Plan Delivery Forms (SPDF), as discussed below. 10. The Performance Management Framework is supplemented by various procedure notes and guidance, including an intranet page dedicated to the people aspect of performance management under HR policies and procedures. There is a brief description of the PM Framework and how the different levels of governance (organisational, departmental, team and individual) goals are aligned so that they are all working towards achieving the same outcome - the Authority s strategic plan. 11. Intrinsic to the new PM Framework is the competency framework, which sets out what is expected of staff in terms of how they do their job. It will also help to ensure that staff have the competencies required to perform their roles and responsibilities, which will determine the achievement of the Authority's strategic plan. 12. From review, roles and responsibilities, and procedures are outlined within the Performance Management Framework. However, it is recommended that these are enhanced. Guidance for staff and Directors to supplement the PM Framework should also be prepared. In addition, the PM Framework should be formally approved. One recommendation has been raised as a result of our work in this area. Performance Indicators 13. The Corporate Health Indicator (CHI) handbook sets out performance measures. The CHI handbook was finalised and formally signed off in September 2008 via a Mayoral Decision (MD) form. The MD notes the Authority's performance against the new set of Corporate Health Indicators and key projects. 14. The Corporate Health Indicators cover: Resources: procurement, finance, ICT, Environment and Sustainability; Workforce: Recruitment and Retention, Learning and Development; Engagement: Listening to Londoners, Equalities and Delivery; and Delivery: Corporate and Operational Business Planning. 15. For each of the indicators, there is a description, rationale, definition, an example (calculation), target, frequency of reporting, and the contact (responsible owner). The CHI s have recently been reviewed to ensure that they will still be fit for purpose in 2010/11. The updated handbook is expected to be finalised and approved by the end of Quarter 4 (Q4), with the intention to start monitoring the new indicator s in the new financial year. 16. Strategic objectives are agreed via Strategic Plan Delivery Forms (SPDF); the deliverables for each project are monitored during the year on a master Item 10 Appendix 1d Page 4

5 spreadsheet which is available on the SMD intranet pages, and is updated quarterly. SMD keep a live version of the document. 17. The SPDFs contain a large amount of information, including project description, purpose, options appraisal, risks, finance, staffing, legal comments and impact assessments. Performance against strategic plan deliverables is the overall responsibility of the Executive Director; the Strategic Plan also names the Mayoral Advisor for each project. A sample of ten SPDF s was chosen from different themes of the Strategic Plan. The sample was chosen from the master outcome database of projects (each relating to a theme) and it was tested whether an SPDF had been completed. It was found that two SPDFs had not been completed from our sample of ten. One was not applicable as the project was largely implemented when the SPDFs were drafted (November 2009) and the other was incorporated into another SPDF. 18. Originally, milestones were not requested to be documented within the SPDFs. Going forward for 2010/11, all SPDFs are to include milestones upon request of the Chief Executive. Evidence of this can be found on the database of actions to be implemented for the Q3 Delivery Review meeting. Therefore, no recommendation has been raised. 19. Furthermore, all projects that require a Programme Budget in 2010/11 are required to detail milestones in their SPDF s; these are referred to as Star Chamber Forms. These forms were used to inform the budget setting process. These forms are more concise than the original SPDFs, but go into more detail in certain areas, for example, they do not just document the cost of the project, but rather the expenditure is broken down so that proposed spend is more transparent and budgets are allocated on more specific information. No recommendations have been raised as a result of our work in this area. Collection, Recording and Reporting 20. Data for Corporate Health Indicators is collected and reviewed either on a quarterly or annual basis. Performance against strategic deliverables (milestones) and CHI s is reported in quarterly Performance Reports that go to the Budget Monitoring Sub- Committee who report back to the Budget and Performance Committee. Detailed testing of the accuracy of CHI data and calculations will be performed in the 2010/11 Performance Indicator audit later in the year. 21. Currently, the method for collecting data for CHI s is being amended. Feeder sheets for each of the indicator sets are being produced, which will allow each of the responsible officers to input the data themselves. The master spreadsheet will then read the values from the feeder sheets. It is set up with worksheets at corporate level (directorates) and directorate level (units). When data is entered into the feeder sheets it is displayed in the master spreadsheet. Performance is calculated and the target cell will turn red, amber or green depending on whether the indicator is ten percent beyond target, within ten percent of target or on/above target respectively. If performance has decreased by five percent in successive months/quarters the calculation cell will turn amber. These two formats will allow easy analysis for reporting to CEDMs. The final worksheets display graphs of the corporate health indicators at varying levels of detail. When data is available year upon year a comparison for the previous year will be displayed (SMDU Process Notes). It is expected that all CHI feeder sheets will be implemented by the end of Q4. Until the master spreadsheet is fully implemented, data continues to be Item 10 Appendix 1d Page 5

6 collected and calculated manually by SMDU and reported on a quarterly basis. The Q1 Performance Report was tabled at the Budget Monitoring Sub-Committee in November All strategic deliverables as identified from the SPDFs are uploaded onto a central database entitled 'all outcomes database', which is available on the Intranet. Every quarter the business coordinator in each directorate collates updates and any additional comments from lead officers for each project. Review of the all outcomes database found that where review of milestones was scheduled to be performed in Q2, there were five projects (out of 88) where there is no evidence of updates being provided. It was confirmed that the Project Support Officer sends out a reminder at the end of each quarter to inform staff that the performance indicator data is due; a couple of reminders could be evidenced. We were informed that the information is generally received within a fortnight. A recommendation has been raised. 23. As part of the PM Framework, a Chief Executive Delivery Meeting (CEDM) is held on a quarterly basis for each Directorate. Various reports are prepared and tabled at these meetings, including budget reports for each cost code, performance against indicators i.e. staffing and invoices, and performance against deliverables. Discussions at the meetings are largely exception based, where actions and performance against targets are discussed. Any actions arising from the meetings are recorded in the database of actions, which is discussed later in the report. 24. The quarterly performance process is finished with the production of a performance report, which includes strategic deliverable and corporate health indicator exceptions i.e. where they have been assessed as red or amber, or below target. Review of the Q1 report found that only one red rating was reported in the report, where review of the all outcomes database showed that there were two red rated deliverables. However, where issues with deliverables flagged red or amber have been resolved between the CEDM and the production of the report, these would not be flagged in the report. One of the deliverables was flagged red as responsibility was split between two directorates, and duplication of reporting was flagged up at the CEDM. This was rapidly resolved, so by the time the Q1 report went out, it was no longer a red rated deliverable, so not included in the report. 25. As part of the PM Framework, a Chief Executive Delivery Meeting (CEDM) is held on a quarterly basis for each Directorate. Various reports are prepared and tabled at these meetings, including budget reports for each cost code, performance against indicators i.e. staffing and invoices, and performance against deliverables. Discussions at the meetings are largely exception based, where actions and performance against targets are discussed. Any actions arising from the meetings are recorded in the database of actions, which is discussed later in the report. 26. It has been noted that the quarterly performance reports are tabled at the Budget Monitoring Sub-Committee four months following the end of the quarter. This delay is due to the Committee s frequency of meetings and timetabling, which needs to take into account time required for Functional Bodies reports to be collated, approved and submitted. However, no recommendation is raised as performance is reviewed and actions agreed on a quarterly basis at the Chief Executive Delivery Meetings (held separately). In addition, the reporting is in accordance with the timing of the Budget Monitoring sub-committee. One recommendation has been raised as a result of our work in this area. Roles and Responsibilities Item 10 Appendix 1d Page 6

7 27. As part of the PM Framework, roles and responsibilities have been set out at each level of governance. However, it is not an exhaustive guide and the Framework states that directorates, units and teams should use this as a basis to tailor their area specific arrangements. Roles and key performance responsibilities have been defined for: Chief Executive Delivery meetings; Directorate Management Team Meetings; Management Team Meetings; and Team meetings. 28. From review of the PM Framework and meeting minutes it was found that reporting lines have been documented and implemented. Directorates are required to report on progress to the SMDU who update the all outcomes database or database of actions. All Executive Directors, Assistant Directors and Heads of Service are accountable for each of the strategic deliverables and report to the Chief Executive during the quarterly delivery meetings. Performance and strategic deliverable exceptions are then reported to the Assembly via the Budget and Performance Committee on a quarterly basis. 29. As documented within the Strategic Plan and PM Framework, exception meetings will take place on an ad hoc basis with the relevant Head of Service and Director if it becomes apparent from the quarterly meetings and reports, that performance is inadequate. These meetings aim to provide analysis of the key issues and to identify improvement actions and timescales for actions agreed. Currently however, there have been no exception meetings that have taken place. Two priority review meetings have taken place, which are designed to discuss specific strategies/projects in more detail. Where priority reviews were highlighted during the CEDMs, these have taken place. No recommendations have been raised as a result of our work in this area. Action Plans 30. A database of actions has been developed by the SMDU and is maintained by the Project Support Officer. All actions have been derived from the CEDMs. Therefore, not all actions are linked to deliverables of the strategic plan, but also on improvements and actions for the SMDU regarding performance management in general. Any other issues arising from the delivery meetings i.e. CHIs, requests for more information (priority reviews) etc, are also recorded and monitored. 31. The action database records the Directorate, in which quarter the meeting was held, the action, owner, team, subject area, RAG (red, amber, green) rating, whether the action is completed or not, and any additional comments. 32. The actions database is a live document. Management can provide information as and when, and the document is updated by the Project Support Officer as necessary and uploaded on the Intranet on a quarterly basis. 33. Review of the database of actions found that where actions had not yet been completed, a comment had been included. At the time of the audit16 actions from Q1 that had not been completed and some were raised again in the Q2 CEDMs. Where issues and actions have been raised again in Q2 meetings, no additional information is provided in the database. In addition, one action is not an action, but an issue: 'Issues around payment of invoices within EfL Team', although the action Item 10 Appendix 1d Page 7

8 is implicit. How this is to be resolved has not been documented; it is the responsibility of the AD/HoU for the team to ensure that an action plan is produced if required, and that the action is completed, and it is recommended that the SMDU be updated on progress. Evidence and discussions need to determine that action is actually being taken. One recommendation has been raised as a result of our work in this area. Lessons Learnt 34. The CEDM is a channel in which feedback can be provided by staff to the SMDU and the Chief Executive regarding issues relating to the structure, monitoring and reporting of Performance. Although this is not a standing item on the agenda, we were informed by the Project Manager and Project Support Officer that staff are encouraged to provide feedback. It is understood that some of this is received during the CEDM, and SMDU meet the Chief Executive at the end of each quarter s meetings to discuss lessons learnt and improvements for the following quarter. 35. We were informed that feedback is often requested, either verbally or by prior to the CEDM quarterly meetings. However, an example could not be evidenced. As no evidence could be obtained regarding feedback on the PM Framework specifically, a recommendation will be raised regarding a formal evaluation of the Framework and the processes for the 2009/10 financial year, from both a SMDU and Directorate perspective. 36. The SMDU is responsible for documenting any lessons learnt from the quarterly CEDMs. Previously, lessons learnt were documented on a separate spreadsheet so that any actions arising could be implemented in the future. However, as it was difficult to distinguish between lessons learnt and actions to be taken, all feedback and changes to be made to the process are now being documented and monitored within the database of actions. This is to simplify monitoring and maintenance of two spreadsheets. One recommendation has been raised as a result of our work in this area. Audit Opinion Substantial Assurance Evaluation Opinion: While there is a basically sound system there are weaknesses, which may put some of the system objectives at risk. Testing Opinion: There is evidence that the level of non-compliance with some of the controls may put some of the system objectives at risk. Item 10 Appendix 1d Page 8

9 Observations and Recommendations In order to assist management in using our reports: We categorise our opinions according to our assessment of the controls in place and the level of compliance with these controls Full Assurance Substantial Assurance Limited Assurance No Assurance There is a sound system of control designed to achieve the system objectives and the controls are being consistently applied. While there is a basically sound system, there are areas of weakness which put some of the system objectives at risk, and/or there is evidence that the level of non-compliance with some of the controls may put some of the system objectives at risk. Weaknesses in the system of controls are such as to put the system objectives at risk, and/or the level of non-compliance puts the system objectives at risk. Control is generally weak, leaving the system open to significant error or abuse, and/or significant non-compliance with basic controls leaves the system open to error or abuse. b) We categorise our recommendations according to their level of priority. Priority 1 Priority 2 Priority 3 Major issues for the attention of senior management. Other recommendations for local management action. Minor matters. Item 10 Appendix 1d Page 9

10 Performance Management Framework 1. Additional detail (Priority 3) Recommendation Rationale The Performance Management Framework should be updated to include at a minimum the: purpose and rationale; Objectives; monitoring and information requirements and processes relating to action plans and the outcomes database; Timeframes; Strategic Plan Delivery Forms; and Corporate Health Indicator (CHI) Handbook. A comprehensive Performance Management Framework will help to ensure that staff understand the Framework and it is embedded into the organisation. A Performance Management Framework is currently in place and available to all staff via the Intranet. However, we found that the document could include more detail about the processes. In particular, with regards to action plans, monitoring and updating the outcomes database and SPDF s. There is an increased risk that insufficient detail results in an ineffective Performance Management Framework; staff unaware of the tasks they are responsible for and failure of the Strategic Plan. Management response: Head of Strategic Management and Delivery The master spreadsheet and actions list are the responsibility of SMD, the performance management framework is an organisational document, and it does not need to have our team s local processes outlined in it we keep a process note for this locally, which was shown to audit. The first two paragraphs in the document essentially set out the purpose and rationale of the document. We are not clear what is meant by timeframes the flow chart sets out what happens when and the table provides further guidance on timing of the meetings. We will be reviewing and expanding the performance management framework as part of a lessons learned project in the next couple of months and will include more detail on the strategic plan and corporate health indicators (to be completed by the end of Q4). Item 10 Appendix 1d Page 10

11 Collection, recording and reporting 2. Timely quarterly updates (Priority 2) Recommendation The Strategic Management and Delivery Unit should ensure that quarterly updates are received for all deliverables in a timely manner. Where lead officers do not provide an update in time for the quarterly Chief Executive Delivery Meetings (CEDMs), these items should be discussed during the meeting and the database updated accordingly. Rationale Providing updates in a timely manner will help to ensure that any underperforming areas are identified in a timely manner and corrective action can be taken. Testing found that progress against deliverables had not been updated by the required timeframe for 5/88 deliverables for the period 2009/10. Where updates are not provided in a timely manner, there is an increased risk that delivery of objectives in a timely manner is not identified and ultimately projects fail, resulting in reputational damage and potential financial loss to the Authority. Management response: Head of Strategic Management and Delivery Any outstanding items are already discussed at the CEDM. We agree that deliverable updates should be received in a timely manner and are reviewing our processes and communications to ensure this happens, as part of the Q4 lessons learned project. Additional Internal Audit Comment: As full minutes of the CEDM meetings are not maintained (the minutes are key action points), we were unable to validate the above comment, however it is agreed that this be tested again during a follow-up audit of this area. Item 10 Appendix 1d Page 11

12 Action Plans 3. Escalation of underperforming areas (Priority 2) Recommendation Where deliverables are red rated or are recurring issues in the quarterly CEDMs, action plans should be developed and exception meetings held in order to identify the source of the problem. Consideration should be given by SMDU to requesting additional information from lead officers regarding the issue and source of the problem, what action is being taken and what progress has been made to date. The database of actions and/or outcomes should be updated accordingly. Rationale Escalation of failing strategic deliverables and underperformance will help to ensure that the necessary action and resources are applied in order to resolve the problem in a timely manner. We found that the two red rated deliverables mentioned previously did not have an action plan documented within the database of actions, and no exception meeting has taken place. Furthermore, where issues have been raised in consecutive CEDMs, no additional information has been provided. Where action is not being taken against failing deliverables and information is not sought, there is an increased risk that underperformance will continue in the long term and may result in the failure of strategic objectives. Management response: Head of Strategic Management and Delivery Disagreed. Exception meetings are held at the request and discretion of the Chief Executive; the decision is based on the reason for the red rating and the priority of the deliverable. A red flagged deliverable in the strategic plan does not automatically trigger an exception meeting, nor would one be a good use of time if the issue can be resolved without. The database of actions reflects the discussion at the CEDM and is not the place to keep detailed action plans. Actions are the responsibility of the AD / HoU for the area, and it is up to them whether they need an action plan or not, in many cases drafting one is not necessary. Additional Internal Audit Comment: Currently there is no evidenced detective control to help ensure that action plans are developed. It is understood from the management comment that the risk is accepted. Item 10 Appendix 1d Page 12

13 Lessons Learnt 4. End of year evaluation (Priority 3) Recommendation Evaluation of the Performance Management Framework should be performed by SMDU at the end of Quarter 4 and reported to the Budget Monitoring sub-committee (and Mayor if appropriate). Feedback should be obtained from all those involved and documented. Rationale Formal evaluation of the Framework by SMDU, including feedback from Executive Directors, Assistant Directors and Heads of Service will help to ensure that improvements from both perspectives will be acknowledged and incorporated where appropriate, resulting in an overall more effective Performance Management Framework. No evidence could be obtained regarding feedback from the key staff involved in the Performance Management Framework. However, it was evident from the database of actions that where improvements could be made following the CEDMs, these were recorded (to be implemented generally by the next quarter). Management response: Head of Strategic Management and Delivery SMDU will be reviewing all of the new performance management processes implemented in 2009/10 by the end of Q4 which will identify lessons learnt, and suggested improvements. This will be based on the feedback we have received during the year, and will go to key stakeholders for comment and review. Any resulting changes to the performance management framework will be outlined in the Q1 performance report MD. Item 10 Appendix 1d Page 13

14 Appendix 1 Audit Framework Audit Objectives The audit is designed to ensure that management has implemented adequate and effective controls within the Authority s Performance Management system, to ensure that key risks associated with the achievement of the systems objectives are effectively managed and controlled. Audit Approach and Methodology The audit approach was developed with reference to an assessment of the risks and management controls operating within each area of the scope. The following procedures were adopted: identification of the role and objectives of each area; identification of risks within the systems, and controls in existence to allow the control objectives to be achieved; and evaluation and testing of controls within the systems. From these procedures we have identified weaknesses in the systems of control, produced specific proposals to improve the control environment and have drawn an overall conclusion on the design and operation of the system. Areas Covered Audit work was undertaken to cover controls in the following areas, to ensure that: The GLA has a documented and approved approach (including policy and procedures) to management of its performance, in accordance with its strategic objectives; Appropriate performance measures have been put in place; Procedures are in place to ensure that accurate and timely performance data is collected, recorded and reported on a regular basis (including scrutiny of performance data by directorates); Roles and responsibilities associated with the monitoring and reporting of performance have been appropriately designated and are clearly defined; Action plans are devised and implemented where there is slippage from performance targets; and Feedback is obtained regarding the Performance Management process and lessons learnt are appropriately communicated. Item 10 Appendix 1d Page 14

15 Appendix 2 - Staff Interviewed We would like to thank all staff that provided assistance during the course of this audit, and in particular: - Senior Projects Officer - Corporate Support Manager - Project Support Officer - Trainee Project Support Officer Item 10 Appendix 1d Page 15

16 Statement of Responsibility We take responsibility for this report which is prepared on the basis of the limitations set out below. The matters raised in this report are only those which came to our attention during the course of our internal audit work and are not necessarily a comprehensive statement of all the weaknesses that exist or all improvements that might be made. Recommendations for improvements should be assessed by you for their full impact before they are implemented. The performance of internal audit work is not and should not be taken as a substitute for management s responsibilities for the application of sound management practices. We emphasise that the responsibility for a sound system of internal controls and the prevention and detection of fraud and other irregularities rests with management and work performed by internal audit should not be relied upon to identify all strengths and weaknesses in internal controls, nor relied upon to identify all circumstances of fraud or irregularity. Auditors, in conducting their work, are required to have regards to the possibility of fraud or irregularities. Even sound systems of internal control can only provide reasonable and not absolute assurance and may not be proof against collusive fraud. Internal audit procedures are designed to focus on areas as identified by management as being of greatest risk and significance and as such we rely on management to provide us full access to their accounting records and transactions for the purposes of our audit work and to ensure the authenticity of these documents. Effective and timely implementation of our recommendations by management is important for the maintenance of a reliable internal control system. The assurance level awarded in our internal audit report is not comparable with the International Standard on Assurance Engagements (ISAE 3000) issued by the International Audit and Assurance Standards Board. Deloitte & Touche Public Sector Internal Audit Limited St Albans April 2010 In this document references to Deloitte are references to Deloitte & Touche Public Sector Internal Audit Limited. Deloitte & Touche Public Sector Internal Audit Limited is a subsidiary of Deloitte LLP, which is the United Kingdom member firm of Deloitte Touche Tohmatsu. Deloitte Touche Tohmatsu is a Swiss Verein (association), and, as such, neither Deloitte Touche Tohmatsu nor any of its member firms has any liability for each other s acts or omissions. Each of the member firms is a separate and independent legal entity operating under the names Deloitte, Deloitte Touche Tohmatsu, or other related names. Services are provided by the member firms or their subsidiaries or affiliates and not by the Deloitte Touche Tohmatsu Verein Deloitte & Touche Public Sector Internal Audit Limited. All rights reserved. Deloitte & Touche Public Sector Internal Audit Limited is registered in England and Wales with registered number Registered office: Hill House, 1 Little New Street, London EC4A 3TR. Item 10 Appendix 1d Page 16