Establishing and Operating a Quality Management System Experiences of the EUROSAI Training Committee Seminar in Budapest

Transcription

1 Workshop Management of an SAI Berlin (Germany), 9-11 April 2008 Establishing and Operating a Quality Management System Experiences of the EUROSAI Training Committee Seminar in Budapest (Dr. Árpád Kovács, President of the State Audit Office of Hungary) First of all, I would like to thank Dr. Engels, President of the Bundesrechnungshof and host of this event for offering me the opportunity to introduce the results of the EUROSAI Seminar on Audit Quality organised in Budapest. Based on my experience I have gained so far, I would also like to take the occasion to congratulate our German colleagues on the excellent and smooth organisation of this important workshop. In the first part of my presentation following a short introduction I would like to briefly summarise why the issue of quality has became one of the most topical questions within the international audit community during the last decade. After this, I am going to talk about the main discussion points of the Seminar. As you can see on the slide, I will speak about the product of the Expert Group on Audit Quality, the implementation of the relevant IFAC standards, experiences related to establishing and improving the quality management system, as well as quality control and quality assurance. Finally, I will sum up the conclusions of the Seminar, and outline what kind of developments we can expect in the near future. In line with the Lima Declaration of Guidelines on Auditing Precepts, the basic function of an SAI is to give a reliable picture to the stakeholders on the utilization of public funds. From the point of view of stakeholders, the acceptance of the audit conclusions and recommendations depends on the trust in the Supreme Audit Institution itself and the audit activities undertaken. Therefore, the SAI should seek to develop systems and methods to assure excellence in their work and outcomes. The process of establishing, implementing and continuously developing a quality management system within an SAI can contribute to ensuring trust with stakeholders. The fundamental goal of setting up such a system is to guarantee consistent good quality of audit work and outcomes. An SAI is in a better position to deliver quality results if it has a properly managed quality management system in place. For a quality management system to function effectively, it is of utmost importance that standards, procedures and guidelines are appropriately defined at all control levels of an SAI. The SAI management assumes the main responsibility for establishing, operating and improving the quality management system. Management concepts identify leadership as one of the key factors for ensuring quality. Leadership styles are well conceptualised and defined in various internationally accepted documents. For instance the ISO standard on Quality Management Principles, the European Foundation for Quality Management Excellence Model

2 and the Common Assessment Framework are in depth dealing with different aspects and functions of leadership. Allow me to mention some thoughts about how the international audit community put more emphasis on quality matters in the last decade when wishing to give adequate answers to the changing audit environment. Let us take the following examples: the INTOSAI Professional Standard Committee has established a project on audit quality; ASOSAI has developed its relevant Guidelines. In addition, one of the main themes to be discussed during the forthcoming EUROSAI Congress, under the chairmanship of the Hungarian SAI, will deal with establishing an audit quality management system within an SAI. Furthermore, under the umbrella of the EU SAIs cooperation network and also supported by SIGMA a special expert group elaborated and published the Guidelines on Audit Quality. Concerning the latter, I wish to note, I am proud that the staff of our office, together with colleagues from the Polish and Maltese SAIs, as well as from the European Court of Auditors, took part in the preparation of the Guidelines and related activities. After becoming aware of these circumstances you may already understand why the State Audit Office of Hungary in close cooperation with the EUROSAI Training Committee undertook, in 2007, the organisation of a seminar focusing on audit quality matters. The Seminar on Audit Quality was held in Budapest, under the chairmanship of the SAI of Hungary and the ECA. Participants from Eastern European countries, the Commonwealth of Independent States, and countries of the Balkan region, as well as presenters from the United Kingdom, Estonia, Poland, Malta, Hungary, Russian Federation, Sweden, and the European Court of Auditors examined matters related to audit quality. In the next part of my presentation I would like to provide you with an overview on the major points discussed during the EUROSAI Seminar in Budapest. The representatives of the Expert Group on Audit Quality which was set up under the EU Contact Committee and supported by SIGMA as well introduced in detail its two basic products, namely the Guidelines on Audit Quality, and the Electronic Database on Audit Quality. The presentation also gave an insight into the requirements to ensure and enhance quality, and technical terms of audit quality. Due to the fact that most of the participants came from countries not being a member of the European Union, this introductory presentation were followed by a vivid discussion on the exact meaning of different terms such as quality assurance, quality control, quality management, hot review, cold review, etc. We had to realize that there was still some scope for misunderstanding. A term can have different shades of meaning in different languages, as a consequence of diverse audit cultures and the way of thinking. Another issue for discussion was the relative importance of the individual components of the quality 2

3 management system. Possibly, I do not even need to mention that both discussion issues were picked up again and again during the seminar. Given the importance of compliance with international standards, and the need to study private sector practices, a guest speaker from the Chamber of Hungarian Auditors was invited to explain how the organization has been implementing the relevant IFAC standards in Hungary. The issue of compliance with IFAC quality standards was also a returning issue, touched upon by many presentations. From the beginning of 2007, every audit firm and individual accountant have been liable for establishing quality control policies and procedures. The Chamber assists the small size firms and individual accountants by giving them guidance. The primary aim of the Chamber is to convince its members that performing quality work is their own interest. During the discussion that followed the presentation, participants came to the view that public and private sector auditing should converge towards each other, from at least the aspect of applying similar approaches. The contribution of the Hungarian SAI during the Seminar presented how the quality management system was set up and how it operates now at State Audit Office of Hungary. The internal regulation concerning the quality management system was also introduced. This document describes and regulates, in a comprehensive manner, factors and conditions having an impact on audit quality; and covers all organisational structures, procedures, processes, and all elements of the audit activity. The Hungarian presentation and the subsequent discussions touched also the problem of how to make colleagues aware of the role of the internal regulation concerning the quality management system. In reality, it was difficult to convince the audit staff to focus continuously on quality issues, and consistently comply with quality requirements, since it cause additional workload for them. Following the introduction of the Hungarian practice, our Russian colleagues also presented their respective experiences. Their quality management system consists of four major components. These are: quality planning, quality assurance, quality control, and quality improvement. Some elements of the quality management system are in place, however their functioning is not yet systematic. Besides emphasising the importance of quality planning, the Chamber considers the other elements of its audit quality management system, namely quality control, quality assurance and quality improvement as equally important for the organisation. For the purposes of quality improvement, the Russian SAI makes special use of the feedback received from the auditees and other stakeholders, such as media, general public, Parliament. In the questions and answers session after the presentation, the main focus was given on the definition of criteria to assess audit quality. Input by participants highlighted various factors such as the timeliness and the usefulness of reports. The acceptance of recommendations by auditees was also mentioned as a possible criterion. Moreover, all participants agreed on the importance of stakeholder feedback when assessing audit quality. 3

4 Let me now summarize the seminar contribution of the European Court of Auditors. The ECA is engaged in the field of quality. They define areas where quality could be added, and implement tools in order to improve the quality of the overall audit activity and outcomes. Besides the audit process itself, the following areas are considered in the course of enhancing quality: resources (including the organisational structure, staff, IT systems and audit standards, etc.), as well as strategic and operational programming activities (which means how to allocate resources in order to achieve the Court s priorities). The ECA representative presented a practical example of how the European Court of Auditors added quality to a specific process. Relevant to this activity, the ECA has elaborated an Audit Guidelines on Evaluation based on the experiences of an audit carried out on the evaluation framework of the European Commission. The guidelines on Evaluation aim at giving auditors an overview of the evaluation carried out, and how it is used and managed by the Commission, as well as how evaluation can be assessed and used as audit evidence in ECA s performance audits. The guidelines also provide a framework to help auditors in exercising their professional judgement in a consistent manner, and in developing programmes for auditing evaluations and evaluation systems. As a conclusion of this presentation, participants pointed out that besides utilising actual experiences and practices when developing methodology and relying on the work of others the following factors support the organisation and its auditors to add quality: capacity and attitude of the organisation and the staff, technical support to implement methodology, knowledge sharing, good practices, and sound professional judgement. Following the general conceptual presentations, more specific topics were presented during the Seminar. The presentation of the Swedish representative addressed the quality issues regarding performance audits. Their performance audits have three milestones; planning, half-way review, and when the draft goes to the auditee. There is an additional language control stage before publishing the report. The key players in the Swedish performance audit quality assurance model are: the head of the performance audit unit, who assumes the overall responsibility for quality assurance; independent quality reviewers, who give a second opinion to the head of the performance audit unit, and write a report at the end of each milestone; the chief operating officer heading the quality department, who give assurance of the overall quality to the competent Auditor General. In my opinion the Swedish example can be considered as a good practice for the following reasons. The model puts special emphasis on the initial phase of quality assurance. In other words, it is important to do right from the very beginning for the purposes of achieving a high quality outcome. Although there 4

5 are mandatory quality assurance criteria at each milestone, the key aim is to provide advice and support concerning quality matters, as well as ensure and enhance mutual learning within the organisation, rather than to hold the audit team to account. The British representative introduced quality procedures that are used by the National Audit Office for financial audits. These procedures of the Office are based on the relevant IFAC standards. According to the British practice there are internal and external quality controls. Internal procedures include: All working papers are reviewed by the audit manager in depth; The competent audit director conducts high level reviews focusing on key risks to giving incorrect audit opinion. In case of high risk audits there is a standards assurance review undertaken by a director or a senior manager independent from the audit team. There is a review carried out upon completing the audit, which is selected based on preset criteria. Such review is undertaken by an independent review team, consisting of experienced directors and/or managers from different areas. As part of the external procedure, the Quality Assurance Directorate of Institute of Chartered Accountants in England and Wales contributes to the assurance of adequate audit quality at the NAO. Feedback from auditees is another element of the external procedure. For the purposes of obtaining feedback, the NAO uses a so-called client feedback questionnaire. Results of quality assurance procedures can lead to changes in the office policy, and key lessons are included in future trainings. During the discussion session, there was a general consensus amongst seminar participants that having high quality staff is key for delivering quality audit work. In this regard it is worth to mention that the UK National Audit Office has framework contract with several private sector accountancy firms to manage shortfalls of staff during peak audit season and to provide expertise in key areas. This practice allows NAO to benchmark performance. Similarly to the British colleague, the representative of the Estonian SAI also addressed the issue of improving audit quality concerning financial audits. His presentation added interesting points and new approaches to the event. In line with the generally followed practice, the Estonian SAI also undertakes both quality control as well as quality assurance activities. They consider as significant prerequisites for carrying out this work the professional competence of its staff, as well as internal written policies and manuals, document templates and auditing software. The implementation of the latter factors makes the reasonable standardization of the audit activity possible. In particular, applying audit documentation software ensures the usage of the same document 5

6 templates in the entire organization. Furthermore, software makes quality reviews easier and provides clear division of responsibilities. The presentation provoked some discussions amongst the audience concerning the usefulness of application of audit documentation software. No consensus was reached because some of the participants were on the opinion that standardisation makes life of auditors easier, while others argued that it binds the auditors in some cases unnecessarily. Towards the end of my presentation please allow me to say some words about the generalisable experiences of the Budapest Seminar. Participants came to an agreement that all types of SAIs need to understand the benefits that can be realised once audit quality is made a priority. As a result of the enhanced quality of the audit work, the contribution of SAIs to the performance and accountability of the public sector will improve. It is needles to say that it is a very comprehensive task. Therefore, the establishment of an adequate quality management system is a long process, and it is necessary to follow a step by step method within the SAI. It was one of the main lessons learnt during the Seminar that the quality is not only an element of the control system within the SAI, it is rather an approach. The staff of the SAI at all levels should be aware of the fact that meeting quality requirements during the audit process and in relation to the audit report promotes performing the basic function of the institution. Right from the beginning, it was obvious for the Seminar participants that improving audit quality requires a systematic SAI-wide approach. Efforts by individuals and individual audit teams are not enough and will not work. SAIs need to proceed methodically and in an organised way to fix and evaluate each quality issue and problem. As new problems will always emerge, this should be a continuous process for the SAI. This is the token of the continuous development of the quality management system. All presenters and participants underlined the responsibility of management at all levels for establishing and operating a quality management system. It was also evident that most audit quality related problems are mainly the result of poor management of the audit process or the SAI itself. Based on Seminar discussions, I would like to mention another important aspect for consideration. In order to ensure the international exchange of information and knowledge, it is essential to use common language. I mean we should resolve problems related to terminology. It is enough to recall the discussions on the meaning of quality control and quality assurance, or in broader sense control and audit. I personally do hope that sooner or later the international audit community will come up with a solution for such cases. 6

7 At the end of my speech, I would like to give you a short update concerning the VII EUROSAI Congress which takes place very soon and also deals with the issue of quality. The President of the Polish Supreme Chamber of Control, the host of the VII EUROSAI Congress invited me to chair Theme 1 of the Congress. Naturally, I was pleased to accept the Polish invitation. The theme covers the questions of establishing and operating an audit quality management system within an SAI. To accomplish the tasks relevant to the elaboration of Theme 1 related documents, a Working Group was set up with the coordination of the Hungarian SAI. It is composed of experts from the SAIs of Denmark, Malta, Poland, and the Russian Federation, as well as from the European Court of Auditors. The Group compiled a Principal Paper, and later a Discussion Paper on the basis of 33 Country Papers submitted. Both papers are discussing the referred theme along the fundamental elements of an audit quality management system, which are leadership; strategy and planning, human resource management; audit process, external relations; continuous development and improvement. I would also like to inform you that the Working Group plans to continue its work after the Congress. Its members are planning to complete a EUROSAI document on good practices of establishing and operating a quality management system within an SAI. According to our wish, the document will contain useful information and hints for those organisations that are striving to provide their stakeholders with high quality audit works and outcomes. The deadline set for the completion of this work is September 2009, that is the forthcoming meeting of the EUROSAI Governing Board. Thank you for your attention. 7