Performance Measures for Internal Auditing A simple question someone may ask is Why measure performance? An even simpler response would be that what gets measured gets done. McMaster University s discussion paper on Performance Indicators spells out several reasons for performance measurement: Clarify the mission and vision of the organization Assist in translating its strategy into measurable objectives Allows the organization to measure progress Understand what improves results Improved accountability and decision making Alignment of operational activates and resources with strategic objectives Encouragement of dialogue A shared understanding of activities planned to deliver objectives Clear communication of expectations to all organizational levels. Internal auditing is also susceptible to performance measurement. The Institute of Internal Auditors (IIA) attribute standards 1311 states that: Internal assessments should include: Ongoing reviews of the performance of internal audit activity; and Periodic reviews performed through self-assessment or by other persons within the organization, with knowledge of internal audit practices and the Standards. On September 29, 2005, the IIA released Practice Advisory 1311-2: Establishing Measures to Support Review of Internal Audit Activities Performance (Appendix A refers). The Practice Advisory recommends a balanced score card approach be used to track performance measurement into three board categories: Stakeholder expectations of Audit Committee, Executive Management, etc Audit process including risk assessment, planning and audit methodologies Innovation and capabilities that include the use of technology, training and industry knowledge. The performance measure selected by an internal group will depend on the resources available to collect the necessary data in systematic way. Some suggested performance indicators are listed in Appendix B. Some of the questions to consider include: How do we make the performance indicator transparent to our stakeholder? How much innovation can we afford? To what end? Are we in a position to establish professional/ national/regional standards in some categories?
Appendix A Practice Advisory 1311-2: Establishing Measures to Support Reviews of Internal Audit Activity Performance Interpretation of Standard 1311 from the International Standards for the Professional Practice of Internal Auditing Related Standards: Attribute Standards: Standard 1311: Internal Assessments Internal assessments should include: -Ongoing reviews of the performance of the internal audit activity; and -Periodic reviews performed through self-assessment or by other persons within the organization, with knowledge of internal audit practices and the Standards. Related Practice Advisory: PA 1311-1 Internal Assessments Nature of this Practice Advisory: This practice advisory expands on PA 1311-1 to provide guidance in establishing measures for ongoing reviews of the performance of the internal audit activity. This guidance is not intended to be the only basis for establishing performance measures but is recommended taking into consideration the International Standards for the Professional Practice of Internal Auditing and common measurement practices. While this advisory provides examples of specific measurements considered critical by CAE s, there is no single set of measurements that is effective for all audit activities. Introduction Standard 1310 provides that internal audit activities should adopt a process to monitor and assess the overall effectiveness of its quality program. The process should include both internal and external assessments. PA 1311-1 suggests using the analysis of performance measures as one method for conducting these reviews. To establish effective performance measurements, the CAE should establish a measurement process that:
Identifies critical performance categories to measure (e.g. internal customer expectations). This advisory suggests the use of the following categories based on a balanced scorecard approach: o Stakeholder Expectations o Internal Audit Processes o Innovation and Capability Identifies performance category strategies and measurements. Strategies are typically based on methods to comply with IIA Standards, other professional standards, and stakeholder expectations Provides an effective ongoing performance measurement and reporting process Links to strategies and includes specific baseline and target measurements to monitor progress. Implementing the use of performance measures, as a method of ongoing review, is one element of the internal audit activity s internal assessment process. Another key element is periodic self-assessment of the internal audit activity s compliance with the Standards. The CAE should ensure that the measures used are appropriate for their activity s size and their applicable industry, country, national laws and regulations, and operating environment. Performance measures should be specific to the organization and the CAE is cautioned against relying on general measures that are not meaningful to the specific audit activity. Examples of measurements considered important to CAE s are listed in a diagram at the end of this practice advisory. Identifying Critical Performance Categories As noted in the introduction, the CAE should identify key performance measurement categories such as stakeholder expectations, audit processes, and innovation and capabilities of internal audit. Stakeholders could include the Audit Committee, Executive Management, external government bodies and regulators, and the external auditors. Audit processes could include risk assessment, planning and audit methodologies. Innovation and capabilities could include effective use of technology, training and industry knowledge. Next the CAE should determine what objectives to meet in each of the above categories, using the mission, strategy, and organization of the activity. It is important to ensure that the applicable IIA Standards and laws and regulations are identified as guidelines for setting strategies. In addition to the IIA Standards, there may be other applicable professional standards.
Identifying Performance Category Strategies & Measurements The IIA Standards, other applicable professional standards, the corporate and internal audit activity mission and strategy, laws and regulations, and the internal audit activity charter will provide an effective foundation for determining the appropriate strategies for each category of performance. Performance category strategies and measurements are based on this foundation and an analysis of stakeholder expectations. Stakeholder Expectations. Typically the key stakeholders for the internal audit activity are divided into internal and external stakeholders. Internal Stakeholders may include: Board/Audit Committee Senior and Operating Management External Stakeholders may include: Regulators External Audit The CAE should identify all relevant stakeholders and perform an assessment of their expectations (and corresponding priorities) and any identified gaps. Assessments can be performed via interviews, facilitated sessions, and questionnaires. As gaps are identified, the CAE is encouraged to develop an appropriate action plan. The expectations among stakeholders may need to be reconciled and validated. Considerations in identifying relevant stakeholders and their expectations include: The extent of regulation for the organization and/or audit activity The relationship with the key internal and external stakeholders The nature of the organization (publicly vs. privately held, for example). Internal Audit Processes. The IIA Performance Standards should be considered in identifying performance measurement categories in internal audit processes. Examples of internal audit process categories and potential areas to measure in each category are: Risk Assessment/Audit Planning. o Are measures in place to ensure that key risk areas are being addressed? o Are measures in place to ensure that appropriate managers are involved in the risk assessment and audit planning processes?
o Percentage of resources allocated to addressing audit committee risk concerns and percentage allocated to addressing senior management risk concerns o Areas where risk concerns have been mitigated/addressed (reduced partially, i.e. from severe to moderate) by effective Internal Audit evaluation and comment o Extent to which audit committee, senior management, and external auditor believe Internal Audit has effectively addressed risk concerns. Planning and Performing the Audit Engagement. o Are measurements in place to ensure that appropriate audit plans are established for each engagement that include scope, objectives, timing, and resource allocations? o Are measurements in place to ensure that audits are performed in accordance with established audit methodologies and working practices? Communication and Reporting. o Is feedback obtained from key stakeholders on the quality, level of detail and frequency of audit communications? o Does the internal audit activity measure the degree that key recommendations are implemented? Innovation and Capability. The IIA Attribute and Performance Standards should be considered in identifying performance measurement categories related to the internal audit activities innovation and capability. Examples of innovation and capability categories and potential areas to measure in each category are: Training. o Are measurements in place to ensure that audit staff receive sufficient training (hours of training per staff, completion by staff of critical training subjects, etc) o Is audit staff satisfaction with training measured? Use of Technology. o Have goals been established for staff training in the use of technology? Are measures in place to ensure these goals are achieved? o Have goals been established for using technology to effectively support audit testing and analysis? Are measures in place to ensure these goals are achieved? Industry Knowledge. o Are measures in place to ensure that the staff has sufficient knowledge of industry, business, operations, and key functions (for example,
measuring the completion of orientation sessions, audit projects in key areas, working in the operations)? Below is a pictorial representation of the concepts discussed above in identifying the performance categories and related measures relevant to the internal audit activity. Internal Customers Board/Audit Committee Senior Management Operating Management External Customers Regulators External Audit Community Corporate Customer Professional Practices Framework Corporate and Internal Audit Strategies Laws and Regulations Innovation and C apabilities Training Technology Industry Knowledge Internal Audit Process Risk Assessment/ Audit Planning Planning & Performing the Audit Engagement Reporting Creating an Effective Performance Measurement and Reporting Process The CAE should establish a measurement process that drives behaviour that supports established audit activity goals/objectives and that provides an appropriate assessment of achievement of those goals/objectives. An effective ongoing process would include: Performance measures that are aligned with the IIA Standards, key strategic objectives, and applicable laws and regulations. These measures can be both qualitative and quantitative. Measurement points should be clear, measurable, achievable, realistic goals and/or standards Consistent processes for gathering, summarizing, and analyzing measurement data and providing timely feedback Processes to ensure measures are kept current with changing expectations, conditions, priorities and objectives Reporting of the results of the measurement process to department management and key stakeholders
Annual reporting on the effectiveness of the internal audit activity to the audit committee. In the example below we have illustrated how the Chief Audit Executive might determine the appropriate performance measurements for innovation and capability: Corporate Strategy Leverage capability of key system s for key financial and m anagem ent reporting as well as autom ating internal controls. Internal A udit S trategy Leverage reporting within key system s for auditing applications that support key processes audited. Perform ance Category: Innovation and Capability Category Strategy: Recruit and train staff in system s reporting and audit c a p a b ilitie s. M easurem ents: - Training hours per auditor on related system s - Num ber of staff with system s audit experience Further information for establishing a framework for measuring the effectiveness of the Internal Audit can be found in: The Quality Assessment Manual published by the Institute of Internal Auditors. A Balanced Scorecard Framework for Internal Auditing Departments by Mark L. Frigo, Ph.D., CPA, CMA. This book was sponsored by the Institute of Internal Auditors Research Foundation. Attached is an exhibit from this book that includes the Most Critical Performance Measures when CAE s were asked to identify the Five Most Vital Measures. 20 Questions Directors Should Ask About Internal Audit by John Fraser and Hugh Lindsay. This paper is available on the IIA website and was sponsored by The IIA Research Foundation, the Canadian Institute of Chartered Accountants, and the Institute of Corporate Directors. On the following page is an excerpt from A Balanced Scorecard Framework for Internal Auditing Departments that lists the measures considered most important by CAE s.
Board/Audit Committee Audit Committee satisfaction survey Role of internal auditing viewed by audit committee Audit Committee risk concerns Management and Auditees Internal Audit Process Auditee satisfaction survey results Percent of audit recommendations implemented Number of management requests Management expectations of internal auditing Number of complaints about audit Professional Practices Framework Corporate and Internal Audit Strategies Laws and Regulations Importance of audit issue Completed vs. planned audits Number of process improvements Number of major audit findings Amount of audit savings Quality assurance techiques developed Number of repeat findings Days from end of field work to report issuance Innovation and Capabilities Staff experience Training hours per internal auditor CAE reporting relationship - functional Percent of certified staff
Some Suggested Performance Measures Appendix B STAKEHOLDER EXPECTATIONS At least 70% of the time is spent on direct audit hours Over a three year cycle, at least 1,000 hours will be spent on: o Compliance audits o Consulting o Information technology audits o Performance measurement o Operational auditing o Risk Management o Value for money audits Fraud Investigations are concluded within 60 days of employee being suspended Client survey is conducted for every operational audit and results summarized for the Audit Committee. INTERNAL AUDIT PROCESSES An in-depth risk assessment will be conducted every three years Risk Assessment profile will be updated annually Over a three-year cycle, some aspect of every department will be audited even if no activity is identified in risk assessment ranking All audit assignments over 50 hours have an audit plan Audit projects are within 10% of the approved budget or maximum of 50 hours over budget Operational Audit Reports are issued within six weeks after exit interview Final reports are issued two weeks after management responses received Management implements over 75% of the recommendations within two years of audit report issuance Management accepts over 90% of the recommendations in the audit reports. INNOVATION AND CAPABILITIES All staff at Associate level and higher will have at least one auditing designation [CIA, CFE, CFI, CISA, etc] At least 10% of the staff time is spent on professional development All staff proficiently use data analysis tools [ACL, IDEA, etc] Staff understand the role of Legislation, Regulation, Policy, directives, and program procedures in relation to employees and public.