Public Service Productivity: Methodology. Metrics of Efficiency and Productivity used by ONS and HMT

Transcription

1 1 January 2008 UK Centre for the Measurement of Government Activity Public Service Productivity: Methodology Metrics of Efficiency and Productivity used by ONS and HMT 1. Objectives of this paper: To clarify similarities and differences in work done by HMT and ONS to measure value for money, efficiency and productivity in public services. 2. Current frameworks for measurement: 2.1 Office for National Statistics (ONS) The UK Centre for the Measurement of Government Activity (UKCeMGA), within the ONS, is responsible for measuring volume of inputs, output and productivity change over time for public services. The aim is to measure all outputs resulting from a broad area of public spending. UKCeMGA contributes to the National Accounts and also publishes productivity articles on specific services. It works within a framework set by the Atkinson Report (ONS 2005). A particular priority is to develop measures for quality of public services in terms of attributable impact on intended outcomes, to include in output and productivity measures. ONS published in July 2007 a strategy, following consultation, for a structured approach to including quality measures as part of output. ONS also work to measure volume of inputs either by using direct methods (e.g. labour) or deflated spending, using service specific measures of pay and price change. ONS do not therefore record improvement in value for money as a result of better procurement as a specific part of their measure of productivity change. The current work programme is on education and children, health and adult social care, social security administration, criminal and civil justice, fire and rescue services and defence. ONS work closely with DCSF, DH, DWP, the

2 Home Office, Ministry of Justice, DCLG, MOD, and also the Scottish Government, Welsh Assembly Government and Northern Ireland departments. 2.2 HM Treasury (HMT) HMT (incorporating the functions previously undertaken by the Office for Government Commerce Efficiency Team) monitors departments progress towards their SR04 obligations. Efficiency looks at the change in the relationship of inputs to outputs at current prices in specific areas of public services. Improvements in this relationship can be claimed as efficiencies where proportionally less resource is used to deliver a particular service and where there is no decline in the quality of relevant services or outputs, or where there are improvements to the quality of outputs. Departments have identified their approaches to measuring efficiency in their Efficiency Technical Notes (ETNs), which are published on their websites. These detail the types of initiatives that are proposed to deliver efficiency gains. HMT monitors progress of those initiatives by tracking delivery of milestones and reports every six months to the Prime Minister and Chancellor. HMT is concerned with tracking two types of efficiency gain, as outlined in the original Gershon Review: cashable efficiencies release financial resources whilst maintaining outputs and output quality, thereby enabling the resources that are released to be redeployed to other services, if desired. non-cashable efficiency gains occur when output volume and/or quality increases, either for the same resource inputs or a proportionately smaller increase in resource inputs in a way that does not release financial resources that can be deployed elsewhere. 3. Areas of commonality in approach 3.1 Performance measurement ONS and HMT are both measuring aspects of the performance of public services. There is a common issue of measuring what is achieved from taxpayers money, and whether society is getting more for less over time. Wider aspects of performance (e.g. reducing inequalities) are measured separately. 2

3 3.2 Frameworks The frameworks used by ONS and HMT are consistent in looking at each of inputs, outputs and quality. 3.3 Scope and Timeframe The frameworks used by ONS and HMT are both primarily national, rather than seeking to measure performance by individual providers or local spending authorities, and both involve comparisons over time rather than benchmarking between management units. 4. Areas of difference 4.1 Scope across public service delivery ONS is currently focused on the themes listed in section 2.1, covering some, but not all the activities falling within the remits of the Departments listed. Where ONS considers a specific public service area for analysis of public service productivity (e.g. social security administration), it aims to capture changes in the entire input and output of that area, adjusted for quality of services. ONS aim to publish figures for the UK as a whole, though it is sometimes necessary to use England figures as a proxy for the UK. ONS will develop methods which enable different patterns for the four countries to be seen, as well as the UK total. HMT work is across all government departments, with a focus on resource usage in specific areas. Where HMT aims to measure an efficiency gain from a specific reform to public services delivery, this is typically done at a greater level of detail than ONS analysis of public service productivity. HMT measures of efficiency do not aim to capture all the inputs, outputs and service quality of a particular sector or sub-sector, but only those relevant to the reform being undertaken, for example in procurement, corporate services delivery and productive time. HMT work covers the policy functions of the Westminster government. 4.2 Effect of price movements Productivity is measured in volume terms whereas efficiency is measured by HMT in terms of current prices. Changes in input costs (e.g. wage and price movements) are deflated out of productivity calculations. This means that a reduction in unit costs of inputs, such as improved procurement practices across government and the wider public sector, are scored by HMT as efficiency gains. If the lower prices mean the public body buys 3

4 more units of input, and use them to provide a higher volume of service, the ONS measure of productivity would not necessarily change. 4.3 Approach to measuring service quality Measures of service quality are present in both the ONS and HMT approaches. HMT measures service quality as a balancing cross check, aiming to ensure that quality of service has not declined in the presence of an efficiency reform. Departments have been responsible for proposing measures of service quality, which were agreed with HMT at the start of the programme. Departments are responsible for monitoring these quality measures and for reporting to HMT where they may be indicating reductions in service quality. ONS define output in a way which includes the quality of service. They aim to make use of measures of quality on the same time basis as inputs and outputs, to judge how quality-adjusted outputs have changed in relation to resources used. This also involves making judgements ideally, research based on the relative importance of different aspects of quality for each service, using the structure of success in delivering intended outcome and responsiveness to users needs. 4.4 Methodology, reporting and decision-making Differences in methodology mean that the requirements of ONS and HMT, and consequently the approaches taken by departments to collect, consolidate and analyse data, may be different. As noted, the level of analysis of ONS and HMT, and the types of management decisions supported, are different. The introduction to the Atkinson Report makes clear that it is seeking aggregate indicators that can form part of the National Accounts, not complete tool kits for the management and audit of government activities. ONS is therefore working on measures that are at a much higher level of aggregation than HMT, where the Efficiency Programme is more concerned with resource usage, often at programme rather than sectoral level. In addition, the policy orientation of the two agendas are different. ONS report retrospectively, for public accountability, with no advance targets or expectations of what productivity change should be. HMT and departments plan actively to set and achieve efficiency targets and build up measures 4

5 from relatively local levels in ways where managers know what they are meant to deliver, and how they are doing. 4.5 Frequency of reporting HMT has an established system for departments to report on their efficiency gains each quarter. Efficiency totals are published with Pre Budget Report and Budget statements, with breakdowns by department available on the HMT website. ONS publishes productivity articles on each service about once a year, but does not have a pre-announced calendar. Articles are prepared with assistance from departmental officials and other experts, but with no Ministerial involvement, and it is up to ONS what to publish. UKCeMGA has an independent Advisory Board; one or more members of the Advisory Board and other subject experts form a quality assurance panel to review each productivity article in advance of publication. 4.6 Remit and timescales HMT current efficiency measurement work is limited to the SR04 period with the remit to ensure that departments deliver their efficiency targets by the end of the SR04 period (end-march 2008). This means HMT s efforts focus on measuring changing levels of spend and delivery in the years up to 2008 compared with a 2004/5 baseline. ONS is working to develop and improve productivity measures through articles that reflect as long a retrospective timescale as possible, but tend not to go before 1996 and sometimes have a later start date. The latest year reported is generally 1-2 years in arrears at the time of the article. Data are normally only reported as annual totals (usually calendar years but sometimes financial years). ONS expect to continue an iterative programme into the indefinite future. 4.7 Audit arrangements The Audit Commission, the National Audit Office (NAO), and regulatory bodies like the Healthcare Commission also have roles in monitoring the value for money of public services. In particular, the NAO has published two studies on the progress made by departments towards the 2004 Spending Review efficiency targets 1. The studies examined the savings 1 The Efficiency Programme: A Second Review of Progress, National Audit Office, February 2007, and Progress in improving government efficiency, National Audit Office, February

6 reported from a sample of projects across the Efficiency Programme. For each project, the NAO assessed the measurement methodology and the data quality associated with both the reported financial gain and the relevant service quality measure. The NAO used a methodology which is broadly consistent with that developed by HM Treasury to monitor the robustness of departments efficiency savings. However, the methodologies differed in two respects: under HM Treasury s measurement guidance, an efficiency gain is sustainable if it can be reported for the current year and two subsequent financial years. In the NAO s view, such savings can only represent sustainable efficiencies if it can be demonstrated that they are part of a programme of reforms which will generate the savings each year as opposed to being one-off savings; while HM Treasury s measurement guidance recognises that accounting for additional ongoing costs arising from reforms is best practice, the NAO considers it essential to net off additional ongoing costs to ensure that reported efficiencies are as accurate as possible. The NAO expects to play a role in providing assurance to Parliament on the value for money savings being delivered by departments in the 2007 Comprehensive Spending Review period. There are no separate arrangements for audit of the productivity calculations in ONS articles, though they are subject to quality assurance before publication. Like all ONS work, they will be due for assessment under the new National Statistics code once the Statistics Act is fully in force. 5. Short to medium term challenges 5.1 Value for money framework in the CSR07 period ( to ) HMT has made some changes in its approach to measures of value for money in the CSR07 period. Departments will be measuring value for money gains and savings, derived from a comparison of actual performance against a counterfactual. HMT will also be taking a different approach to service quality and relying on the new PSA/DSO indicator framework to ensure that the government s priorities are still being delivered in a tighter spending round. And all savings will be net of costs. Once the CSR period is underway, ONS and HMT will discuss again how their measures of effectiveness, efficiency and productivity relate, and how 6

7 they might both work with spending departments in a clear overall framework which maximises useful analysis and accountability from the available data and staff resource. Note This paper was prepared by UKCeMGA in ONS in cooperation with officials in HMT and NAO. 7