7 Chapter 17 Performance Monitoring Performance Monitoring Contents: 17.1 Introduction 17.2 A Wider Perspective 17.3 Requirements for a Good Performance Indicator 17.4 Types of Performance Monitoring 17.5 Arrangements for Collecting Data 17.6 Transportation Service Performance Indicators 17.7 Regular Monitoring, Reporting and Balanced Scorecard 238
Chapter 17 Performance Monitoring 17 Performance Monitoring 17.1 Introduction Surrey County Council has developed a mission statement and a broad number of objectives. At this very highest level, these can provide an appropriate basis for the development of a suite of performance measures to measure our progress. Below this level our Service has developed its vision for the future and its own priorities within a corporate performance framework. The concept of performance wheels has recently been introduced to provide a basis for measuring service delivery throughout the authority. Our performance wheels focus on four key drivers: our customers; our operations; our staff and members and our resources. The performance wheels operate in a complementary way, on a hierarchal corporate, service and group level. Asset management has strengthened the concept of Levels of Service and it is important to demonstrate the relationship between these and the various priorities we have identified within our Service Performance Wheel. Our asset management plan sets out a rigorous approach for monitoring our progress towards these various priorities. It does this by collating asset related performance measures matched to specific objectives and levels of Service, in order to provide us with focussed information on our progress towards these. The national framework makes the following comment about linking performance indicators to Service objectives: The important consideration, in starting by identifying the objective first, is to establish that the elements of the Service being measured genuinely contribute to the delivery of a strategic goal. This helps to develop an understanding of how the system of service delivery is, or is not, supporting progress towards achieving established strategic goals. The following diagram provides a simple example of this linkage, using an existing Service wheel priority and a Level of Service proposed within this plan. Level of Service Service Wheel Level of Service Performance Priority/Objective Subgroup Indicator Condition of the asset Improve our roads Road condition BVPI 97A % of nonand their environs principal classified road network where minimum threshold level is not met. Our Transportation Service has identified a range of performance indicators to monitor its performance, and manage its progress. These have been collated from a number of sources: National Best Value Performance Indicators National Code of Practice for Maintenance Management (Appendix F and G) Surrey Highways Partnership Contract key performance indicators South East Counties Service Improvement Group Drawn up by the Transportation Service (Performance Wheel Supporting Table) Performance indicators derived for specific purposes (e.g. Street lighting contract) 239
7 Chapter 17 Performance Monitoring There is currently a national programme to expand the range of local performance indicators for selective use by local highway authorities. The aim is to provide indicators for structures and lighting maintenance, network management and road safety in addition to those currently covering highway maintenance. It will be advisable to utilise these where appropriate within our suite of indicators. This will provide us with a measure of our progress against other local highway authorities through this common set of indicators. In broad terms, performance measures can be divided or used in the following way: Strategic measures providing a measure of annual performance to external stakeholders. These do not assist with the day-to-day management of the asset. These are the national best value performance indicators. Tactical measures providing us with management information, including general and specific asset condition to assist in future programming and resource allocations. Operational measures principally designed to assist in the improvement of service delivery. Typically, these are provided for closer and more regular monitoring and will include a number of our contract indicators. In general, asset management measures are focussed on the tactical issues. However, while asset management indicators are more likely to be tactical in nature, it will be important to maintain a balanced approach across the whole range of performance measures, in the future. 17.2 A wider perspective The Code of Practice for Maintenance Management highlighted the importance of performance management and its role in achieving continuous improvement. As part of this process, it was important to have regard to three key aims: economy; efficiency and effectiveness. The Audit Commission defined these terms as: Economy acquiring human and material resources of the appropriate quality and quantity at the lowest cost. Efficiency producing the maximum output for any given set of resource inputs, or using the minimum inputs for the required quantity and quality of service provided. Effectiveness having the organisation meet the citizen s (customer) requirements and having a programme or activity achieve its established goals or intended aims. The Code refers to the fact that national best value performance indicators have been categorised within one of five broad dimensions. It is recommended that these are considered in the development of local versions: 240
Chapter 17 Performance Monitoring 17 Cost and efficiency Effectiveness Quality of service (including customer satisfaction) Ease and equality of access Strategic objectives of service provision to show that the Service is being provided at a price comparable to what other authorities are achieving of service delivery and management to show that effective systems are in place to deliver the service to show that the quality of service delivered is commensurate with the price paid to show that stakeholders/customers have easy access to the services they need regardless of gender, race etc to show that the services provided meet the needs The Code of Practice has indicated that these dimensions embrace all the requirements of an indicator. For this reason they should be given appropriate consideration in the future development of all our future local performance indicators. The Code points out as well that the emphasis on our local performance measures should be on their importance and relevance to our customers. While aimed at highway maintenance, these messages have equal relevance for all work on our asset. 17.3 Requirements for a Good Performance Indicator There have been many attempts to provide people with the preferred ingredients of a good indicator. However, for the purposes of this plan the opportunity has been taken to highlight the advice given in the national framework. Ideally a performance measure should be: Objective: this is desirable but some subjective measures will be inevitable Repeatable: should be able to be repeatedly measured, with appropriate accuracy. Reproducible: should be able to be reproduced by a different operator and instrument. Aligned with objectives: should be linked as closely as possible to our Service priorities. Cost effective: the cost of collecting the data must be reasonable when compared to the perceived benefit. Manageable: should be able to be controlled or influenced by Service deliverers. Able to be modelled: ideally it should be possible to predict the change of a measure with time using some form of model e.g. pavement deterioration. Safe to measure: ideally measured using high-speed data collection. 17.4 Types of Performance Monitoring The national framework describes a number of the more commonly used arrangements for measuring performance. These are: Monthly where established management systems are in place it is possible to obtain monthly performance statistics. In general, these may focus on activities and the time taken to complete them. Progress of work against established programmes may also provide a useful management tool. Annual some indicators will be based on information supplied by others annually. This limits the use of such indicators to annual reviews. 241
7 Chapter 17 Performance Monitoring Random auditing* audits can include customer satisfaction surveys (MORI), sample condition surveys and adhoc inspections. Random audits are frequently used to independently audit the performance of highway authorities that are working under quality management systems. Their purpose, therefore, is to provide an independent crosscheck on performance. * Reference should also be made to Programmed Auditing, which would refer to the use of CVI, DVI and CONFIRM data to provide us with easily accessed, regular monitoring information. System audits* where asset management software systems have been implemented the system, if properly used, will provide a source of management performance data. For example, customer query systems will be able to provide response times to queries and also be able to identify areas of exception. These might include particular types of complaint or query, or particular geographical areas generating high numbers of queries. * Use of the CONFIRM system to generate a coarse assessment score (1 to 10) of carriageway and footway visual condition for monitoring random sample sections. In addition to the more common ones referred to in the framework, monitoring can also be undertaken through: Quarterly returns some indicators will be reported on a quarterly cycle where it is more appropriate to measure on this basis and provide interim information to adjust performance to meet year-end targets. Compliance monitoring will include the monitoring of constructor and consultant compliance with contract requirements including response times, quality of decisions, quality of work etc. It will also include systematic coring of statutory undertaker reinstatement works in order to seek improvements in quality, and minimise the adverse affect on our asset. It may also be beneficial to develop further performance audits, which might include: Network Condition Auditing (Hertfordshire CC arrangements). A random sample of network lengths are reviewed on a monthly basis, and the appearance of certain aspects of asset elements within the network are visually graded. The grading is recorded against specified standards, and the information used to calculate various condition indices for physical assets. Service Inspections regular, mainly specialist inspections, allowing a monitored record of condition (against set standards) of certain sites within the programme of inspection, e.g. safety fence. Network Integrity Inspections carry out engineering level service inspections to review the integrity of the overall asset and to use this information to monitor overall improvements to the network. 242
Chapter 17 Performance Monitoring 17 17.5 Arrangements for Collecting Data It is important that we get the full commitment of colleagues in reporting performance data. Previous attempts to establish performance monitoring regimes have had limited success due to a lack of commitment from those who are tasked with collecting and monitoring the information gathered. The national framework points out that it is important to ensure that, where possible: Data is gathered as an integral part of carrying out work, rather than as a separate task. Data is collected by the people who need to use it to make decisions. It is important to check the validity of the performance data to ensure that these results are genuinely reflecting the current performance. The national framework also draws attention to problems associated with the level of accuracy in relation to target values. Problems are likely to occur when the difference between the target level and the measured condition is small, and the accuracy of the measuring method may well be the reason for non-achievement rather than the actual performance provided. Taking this advice onboard, it is important that we minimise the additional work required to collect our performance data. We will need to ensure that data is provided by our Constructors as an integral part of their operations, and that we adapt our systems, particularly CONFIRM, to provide easily accessible reporting data. 17.6 Transportation Service Performance Indicators Part of the work of the Steering Group has been to identify and collate the range of current and future asset related indicators operating within our Transportation Service. These have been drawn from a number of sources, principally the Transportation Service Performance Wheel Supporting Indicator Table for 2004 to 2007. However, there are also a number of other performance indicator sources, and these are: Surrey Highways Partnership Contract National Best Value Code of Practice (Appendix F and G as appropriate) South East Counties Service Improvement Group (SECSIG)- some remaining i.e. highway safety indicators Individual Transportation Group sources These various indicators have been collated, any gaps identified and any additional indicators added, as necessary. Appendix I to this Plan provides a schedule setting out each of the six identified Levels of Service, the associated asset related Performance Wheel Priorities, the Levels of Service sub-groups, together with the associated performance indicators which each provide a measure of our progress towards our Service Priorities. This provides us with a structured framework to measure progress against our various objectives. To enable us to judge our progress on a number of the nationally promoted local performance indicators, when compared with other local highway authorities, we are now a member of a national performance benchmarking club. 243
7 Chapter 17 Performance Monitoring 17.7 Regular Monitoring and Reporting and Balanced Scorecard The following advice is drawn from the CSS Framework for Highway Asset Management. There are a number of ways in which performance measures can be summarised and reported. In deciding upon a reporting format, consideration should be given to: Showing the alignment between the measures and the outcomes/strategic goals Reflecting a balance between competing demands Presenting only information that the audience requires Keeping the number of measures to a manageable level Increasingly, there is a demand for organisations to report on not only the financial performance of their assets, but also on the social and environmental effects of their actions. An increasing number of organisations worldwide are adopting a triple bottom line reporting regime. Under such a regime the outcomes of their actions are reported against the social, economic and environmental outcomes. For purely commercial organisations this means a move away from a purely financial focus to being able to demonstrate to shareholders that they are also managing their social and environmental responsibilities. Clearly, there is an inherent tension between these competing demands. A balanced scorecard approach is therefore appropriate as a tool for demonstrating the level of attention given to each outcome area. Developing a scorecard requires input from a variety of stakeholders and will inevitably produce a tension between conflicting demands. It is recognised that we do not currently produce scorecards such as that shown below. The steering group have identified this as an improvement action, as part of the Improvement Plan. It is important to bear in mind the concept that if it is not measured it cannot be managed. 244
Chapter 17 Performance Monitoring 17 Health Scorecard: Sample Score 82.3 245