Key Players in Performance Management & Performance Measurement

Transcription

1 Key Players in Performance Management & Performance Measurement William Dorotinsky October 11-12, IAS Conference: Performance Audit by Public Sector Internal Auditors

2 Table of Contents Objectives of performance systems Useful concepts and constructs Users & uses of performance information Performance models & techniques Pressure for results Role of internal auditors Concluding remarks 1

3 Objectives of Performance Systems Performance systems emphasize various objectives: 1. To learn from Tools: strategic planning, internal management information systems, benchmarking, risk analysis, business process reengineering Primarily for internal ministry management Examples: US Statestat, Dutch benchmarking in sectors 2. To steer and control, align efforts to goals Tools: monitoring and management scorecards, performance pay, strategic performance budgeting Primarily for central government (MoF/Budget, PM etc..) Examples: Japanese program and policy evaluation, South Korea performance budgeting 3. To provide accountability Tools: citizen charters, annual reporting, performance contracts Primarily for central government (MoF/Budget, PM etc..) Primarily for reporting to citizens, media, external stakeholders Examples: UK Charters/Compacts, Peru s RECURSO program, US Expectmore.gov Value is in: Strategic planning Focus on longer term objectives Align components towards same objectives Enable coordination towards objectives Performance monitoring Monitor progress towards objectives Identify areas off-track Manage off-track areas to move them forward Types are not mutually exclusive, but reflect the relative emphasis present in any given system. 2

4 A Complex Transition in the Public Sector Control & compliance o Rules governed o Limited flexibility o Non-preventive o Negative focus o Control of individuals Performance & Accountability o Mission driven o Customer driven o Employee responsive o Flexible o Preventive o Positive team focus Gauging Interests A significant cultural & institutional change Performance is for the most part determined by where you sit As a purchaser of a service, your interests are typically different from those of the supplier of the service o Buyer of car; versus manufacturer of car As a regulator, your interests are typically different from those of the regulated o Compliant behavior versus lightness of burden In some cases, parties may have interests as both purchasers and suppliers (e.g. co-operatives) Government performance is complex in part because of the variety of interests Standard problems of government production With different degrees of clarity, performance systems attempt to address these various interests Institutional fabric is important o Westminster; Presidential; other Parliamentary etc o Common vs Civil Law 3

5 Evolution of Control Approaches Ex ante (to commitment) Ex Poste External (to spending unit) Internal Centralized commitment control (transaction approval) Allocations (commitment limits) Warrants (cash limits) Procurement procedures Personnel/pay rules continuous auditing Disbursement rules Ministry or spending unit transaction approval Procedures to minimize risk (internal controls) Transparency Central internal audit External audit Regular reporting, management intervention Quarterly close-outs Cash rationing Transparency Ministry internal audit Performance management 4

6 Concepts & Constructs A production model of performance: Environment Final Outcomes Needs Socio-economic Situation Relevance Intermediate Outcomes Effectiveness Organization or Programme Objectives Inputs Activities Outputs Efficiency Economy Cost Effectiveness Utility and Sustainability Source: Wouter van Dooren, Geert Bouckaert, John Halligan, Performance Management in the Public Sector (2010) 5

7 The Relationship between Performance Measurement & Management Supply of performance information Demand of performance information Measurement Incorporation Use Source: Wouter van Dooren, Geert Bouckaert, John Halligan, Performance Management in the Public Sector (2010) 6

8 User Across the Public Sector: Users & Uses of Performance Information Potential uses of performance information Key steerering agent Presidency/Cabinet Commonly set key objectives and policy priorities for the government Ministry of Finance/Budget Line Ministries & Providers Legislative/Parliam ent External Audit Public Reviewer of rowers performance Prepares consolidated budget submission with increasing use of performance information to justify allocations Monitors within year performance Typically guide performance reforms with firm regulation and oversight Key rower Generate majority of performance information Tend to use performance information for internal management, but are beginning to use it in negotiating budgets with the executive Key steerer or reviewer, depending on characteristics of system Access to increasing amounts of performance information, mixed evidence of use in the approval or review of the budget Key attester of financial and increasingly, non-financial information Increasing use of performance audits and program/sector reviews May also conduct audits of performance information systems and performance measures/indicators/targets Key consumer, taxpayer and voter Experiential information, formal use of performance information has been fairly limited, although interest groups, Think-Tanks and the media focus on areas, such as league tables/citizen charters in service areas, e.g. school exam scores, hospital treatment waiting lists etc Source: Adapted from T. Thomas/J.Brumby 7

9 User Within Ministries: Users & Uses of Performance Information Potential uses of performance information Management of programs, real-time readjustments Program Managers Quick assessments of impact for learning what to do better Reporting Senior Officials in central agencies & ministries Internal Audit (Potential Roles) Ministers Assessing performance of programs across the Ministry Managing entity performance Reallocating resources (time, $, staff) across programs Tracking key priorities Performance Audit Ensuring use/incorporation of performance information into decision making Verification of quality of performance information Assessing relevance of performance tools in place Conducting performance assessments Holding staff accountable for performance Reporting upward Communicating with the public/clients 8

10 Performance Models & Techniques Different performance tools have been developed to meet different objectives and problems, and address different linkages and accountability arrangements among various actors within the public sector. MoF/MoE/ Treasury Ministers Legislature Public Service Agreements (UK PSAs) Performance Contracts (NZ, Denmark) Executive Ministries Ministries Front Line Service Providers Balanced Scorecards (US, Thailand) Performance Monitoring (Brazil) Project Management Models Benchmarking Performance Auditing (SAIs) Within Ministries CAF BSC Front Line Service Providers Citizens Client Satisfaction Surveys (Philippines) Citizen Charters (Kenya) 9

11 Pressure for Results hypothetical External Audit Prime Minister s Office Ministry Department Internal Audit Cabinet Civil Service Ministry Media/ Citizens Planning Ministry Front Line Legislature Finance Ministry The primary use and benefit of performance information resides within Government ministries & departments for management purposes. Actors external to this environment exert pressure for accountability and control. 10

12 The Role of Internal Audit Internal Audit s scope within an organization can be broad and may involve topics such as: the efficiency of operations the reliability of financial reporting deterring and investigating corruption compliance with laws and regulations IT audits risk assessments performance auditing etc Definition of Internal Audit An independent, objective assurance and consulting activity designed to add value and improve an organization's operations. It helps an organization accomplish its objectives by bringing a systematic, disciplined approach to evaluate and improve the effectiveness of risk management, control, and governance processes. Source: The Institute of Internal Auditors In most countries, performance audits of governmental activities are carried out by the external audit bodies, but performance audits may also be conducted by internal auditors. 11

13 Performance Auditing Performance audit -the independent examination of the efficiency and effectiveness of government undertakings, programs or organizations, with due regard to economy, and the aim of leading to improvements. In most countries, performance audits of governmental activities are carried out by the external audit bodies, but performance audits may also be conducted by internal auditors. Performance auditing vs. Performance monitoring Ex-post Time lag in feedback Conducted by internal or external auditor or evaluator Continuous Immediate feedback loop into operations Management responsible Source: INTOSAI, Implementation Guidelines for Performance Auditing 12

14 Key Questions for Internal Auditors in the Performance Space Internal auditors may also consider assessing the quality of performance management & measurement that is occurring at different levels of an organization. Key questions include: - Are managers being held accountable for performance? If not, why? - What factors may be limiting individual accountability? - How is performance information being used? What incentives exist to produce, incorporate, and use performance measurement? - What is the quality of the performance information being produced at various levels? - What impact are the performance management tools (e.g. Balanced Scorecards) having on the entity? 13

15 Systemic Obstacles to Unleashing Performance It is difficult to hold managers to account if: Unclear organizational structure, mandate They were not involved in budget setting, planning, target setting Performance metrics, targets change annually Do not actually receive approved budget levels; no predictability in funds availability Have no influence over procurement processes for their work Have no influence or human resource management for their office (positions or employment) Do not receive continuous or regular spending reports for their units No regular management or performance and output reports Have no flexibility or discretion in resource allocation Have treasury/accounting systems on a cash basis, necessitating their own parallel commitment accounting Managers selected for technical skills, not management ability No managerial training, support system Auditing focuses on compliance, while legislature and chief executive focus on performance results Internal Audit could offer recommendations on systemic/structural impediments to performance. 14

16 Misalignment of Various Systems HR Controls Budget Controls Performance Pay Planning Level Reporting on Performance Financial Reporting Ministries Departments Offices Unit Teams Individuals 15

17 Foundation for Managing Performance Entities need to know what they are supposed to accomplish Tougher than it sounds Valid measures of performance need to exist Tougher on the non-observable outcomes and outputs Accurate measures of cost need to be developed Cash and accrual Systems requirements Cost and performance information should be brought together in budget process Per stages identified Participants need to have incentives to use the information Probably as important as any aspect Urban Institute 2001 neither executive or legislative branch systematically used outcome data for budgeting 16

18 Concluding Remarks Understanding performance models & tools: What are the uses and who are the users of each model? How is the impact of the model measured? What is the broader environment in which these models will operate? How is the quality of the performance information generated from these models assured? What is the unit of analysis? (Ministry, Program, Project, Individuals, Departments) Bigger questions to keep in mind during the day: What role should Internal Audit have in the performance agenda? Under what circumstances should IA assume such a role? Key debates in incorporating a performance orientation for Internal Audit: getting the balance right between compliance & performance new skill requirements legal mandate, objectives/roles, reporting relationship to External Audit avoiding duplication in efforts, aligning with other performance assessments 17

19 Efficiency and Economy of Performance Monitoring? The Worker The man who monitors the worker The man who monitors the man who monitors the worker The man who monitors the man who monitors the man who monitors the worker The man who wrote the draft policy guide on how to monitor the man who monitors the worker 18

20 Thank You. 19

21 Additional Slides 20

22 Thailand s Performance Tools a bird s eye view OPDC BOB OCSC NESDB DOLA BSC/PMQA PART HR BSC Value for Money Service Standards/ LQM 20 Ministries 141 Departmen ts 18 Provincial Clusters 75 Provinces 73 Academic Institutions Local Administr ative Org. 21

23 Outputs and Indicator Characteristics Quantity Quality Cost Timeliness Location Externally focused Controllable Comprehensive Measurable Informative the quality or standard of output expected the quantity of output to be delivered the cost of output delivery the timeframe for delivery of the output the physical location where the output will be delivered The output describes a distinct product or service. Output specifications appropriately reflect the extent of control over the delivery of each output. The outputs should incorporate all of the core outputs that were delivered by the entity. This includes outputs where delivery has been subcontracted to another party. It also may include all outputs funded by parties other than ratepayers or taxpayers for example, outputs funded through user charges. Output specifications provide clear descriptions and standards against which actual performance can be judged. Output descriptions are sufficient to meet the information needs of a relatively new or unfamiliar user of the information. 22