T HE EUROPEAN COURT OF AUDITORS T HE DAS M ETHODOLOGY

Transcription

1 T HE EUROPEAN COURT OF AUDITORS T HE DAS M ETHODOLOGY E N G L II S H

2 This brochure introduces the Court of Auditors focusing primarily on the annual Statement of Assurance issued by the Court, generally referred to as the DAS and provides an insight into the methodology that is followed by the Court in carrying out its audits and in its formulation of the DAS. It must be noted that this brochure does not cover the subject of sound financial management which the Court addresses through a number of value for money audits, the results of which are published in the form of Special Reports.

3 Table of contents THE EUROPEAN COURT OF AUDITORS 3 Mission Statement 3 Mandate 3 Statement of Assurance 3 THE CONTEXT 4 TWO FACETS TO THE DAS 5 An audit of the accounts 5 An audit of the legality and regularity of the underlying transactions 5 THE DAS - FROM MANDATE TO METHODOLOGY 6 A brief history 6 Anticipating further change 6 Choice of methodology 6 International auditing standards 6 RELIABILITY OF THE ACCOUNTS 7 MAIN SOURCES FOR THE AUDIT OF LEGALITY & REGULARITY 8 Relation between the different sources 8 Supervisory and control systems 8 Substantive testing 9 Annual Activity Reports and other auditors reports 9

4 THE DAS AUDIT PROCESS 10 Materiality 11 Risk analysis 12 The audit cycle 12 Reasonable assurance 12 Confidence level 13 Assurance model 14 PERFORMING AUDIT PROCEDURES 15 Examination of supervisory and control systems 15 Substantive testing 15 Analysis of Annual Activity Reports & declarations by the Directors-General 19 Other auditors work 19 Dealing with differences in the findings obtained 20 FORMING AUDIT CONCLUSIONS & AN AUDIT OPINION 21 Specific assessments 21 Statement of Assurance 21 Improvements in systems 22 GLOSSARY 24 FREQUENTLY ASKED QUESTIONS 28

5 The European Court of Auditors The European Court of Auditors (ECA) is one of the European Union s (EU) Institutions. It is the external auditor of the EU finances and institutions, an essential role in a modern democratic society. Mission Statement to audit independently the collection and spending of EU funds and, through this, assess the way that the European Institutions discharge these functions; to examine whether financial operations have been properly recorded, legally and regularly executed and managed so as to ensure economy, efficiency and effectiveness; to make known the results of its work through the publication of relevant, objective and timely reports; to contribute to improving the financial management of EU funds at all levels, so as to ensure maximum value for money for the citizens of the Union. In particular, under the provisions of Article 248 (1): The Court of Auditors shall examine the accounts of all revenue and expenditure of the Community ( ) The Court of Auditors shall provide the European Parliament and the Council with a statement of assurance as to the reliability of the accounts and the legality and regularity of the underlying transactions ( ) This statement may be supplemented by specific assessments for each major area of Community activity. The Court is also required to examine whether the financial management of EUfunds has been sound i.e. carried out with due regard for economy, efficiency and effectiveness. This particular aspect, however, is not covered within this brochure. Statement of Assurance The primary aim of the DAS is to provide stakeholders, mainly the European Parliament and the Council of Ministers but also EU citizens in general, with an appraisal and opinion regarding the way in which EU money has been spent. This Statement of Assurance is generally known by its French acronym DAS (Déclaration d Assurance). Mandate The mandate and composition of the ECA are set-out in articles 246 to 248 of the Treaty establishing the European Community (EC Treaty). 3

6 The context No audit task should be considered as being easy. However, the EU context makes the task of the European Court of Auditors particularly challenging. This represents a significant methodological and logistical challenge when conducting an audit because of the large number of systems and subsystems and transactions to be covered. The following are 3 important aspects. Around 80% of the budget is subject to shared management of expenditure between the Commission and the Member States. The different ways in which the Community budget is implemented (direct and indirect centralised management, shared management, decentralised management, joint management) and the complexity of its component systems makes the DAS audit quite particular. Over 80% of expenditure concerns payments made on the basis of claims submitted by many and diverse final beneficiaries in the Member States and in third countries. There is a high level of complexity in the rules applied in the implementation of so many different expenditure programmes in the 27 Member States. It is a characteristic of the EU budget that financial transactions involve several administrative levels. In general, payments are made by the Commission to the Member States, which then make payments to their relevant agencies and/or to different regional authorities. Effectively, these act as intermediaries and make payments to the final recipients, often via additional stages in between. 4

7 Two facets to the DAS The DAS represents the formal opinion of the ECA on the reliability of the EU accounts and on the legality and regularity of the underlying transactions. Within this context, the DAS has two facets, as follows: An audit of the accounts The opinion on the reliability of the accounts states whether the final consolidated accounts of the European Communities for a given year completely and accurately report the cash flows and financial results for that particular year and assets and liabilities at the year end are properly registered in order to reflect faithfully the financial position. The annual accounts comprise the consolidated financial statements covering, in particular, the balance sheet setting out the assets and liabilities at the end of the year and the consolidated reports on the implementation of the budget covering the revenue and expenditure for the year. In this aspect of its audit, the Court sets out to establish whether the European Commission has ensured the effective application of the relevant accounting rules and whether the consolidated final accounts give a true and fair view of the EU finances within the framework of the generally accepted accounting principles and methods. The approach followed by the Court in the audit of the accounts of the European Communities follows standard financial audit methodology, i.e. the following basic elements: the evaluation of the central accounting system; checking of the functioning of the key accounting procedures; analytical check (consistency and reasonableness) on the main accounting data; analyses and reconciliations of accounts and/or balances; substantive tests on commitments, payments and certain balance sheet items. An audit of the legality and regularity of the underlying transactions The objective is to gain sufficient evidence as to whether funds have been received and spent in conformity with contractual and legislative conditions and have been correctly and accurately calculated. In brief, this involves the determination of whether the transactions took place, whether the recipients/beneficiaries were eligible for the funds received, and whether the costs/quantities claimed were accurate and eligible. The resulting audit opinion consists of an overall conclusion covering all revenue and expenditure transactions. Payments are audited down to the level of the final beneficiaries. The DAS is derived on the basis of audits carried out covering the year preceding the publication of the ECA s annual report, within which it is the central element. 5

8 The DAS - from Mandate to Methodology A brief history With the Treaty of Maastricht coming into effect at the end of 1993, the DAS was implemented for the first time for the 1994 financial year. The European Court of Auditors Annual Reports started to contain a Statement of Assurance, certifying the reliability of the accounts and the legality and regularity of the underlying transactions. Anticipating further change Faced with the dynamically changing environment and given past experience and lessons learnt over the last decade, it was necessary to develop and adapt the underlying DAS methodology. The key elements of the recently revised approach adopted by the Court in February 2006 are described in the present document. For example, due to the implementation of Activity Based Budgeting and Management (ABB/ABM, integrating budgeting and financial management) by the Commission, as of 2007 the Court will not issue specific assessments based on the financial perspective areas, but on clusters of ABB policy areas. Choice of methodology The revised DAS methodology takes into account a number of key factors: International auditing standards; Past experience; The position of the principal DAS users, the European Parliament and the Council; An open dialogue with the main auditee, namely the Commission; Resource constraints. International auditing standards The process of developing an appropriate methodology needs to be flexible in order to accommodate future changes. Certain specific issues, in particular the implementation of the new financial perspectives and the reform of Community policies, might make it necessary to further update the approach as the need arises. Professional ethics require complying to the largest possible extent with generally accepted international auditing standards. Existing international standards essentially cover the audit of the financial statements of private companies or public entities, and only in exceptional cases provide guidelines concerning the legality and regularity of underlying transactions. Therefore, most of the international standards can only be applied by analogy. 6

9 Reliability of the accounts The opinion on the reliability of the accounts sets out to establish the extent to which the final consolidated accounts of the European Communities for a given year completely and accurately report the cash flows and financial results for that particular year and assets and liabilities at the year end. Therefore, in this aspect of its audit, the Court sets out to establish whether the European Commission has ensured the effective application of the relevant accounting rules and whether measures taken by the Commission contribute to the reasonable assurance that the consolidated final accounts are true and fair. The objective is to gain sufficient evidence that all transactions, assets and liabilities have been completely, correctly and accurately recorded in the accounting records and presented in the financial statements. As from 2005, the consolidated accounts of the European Communities have been produced following the principles of accruals based accounting. This has been a very important development. The introduction of accruals based accounting shifts the focus of the accounts from the simple registration of cash transactions to the recording of inflows and outflows of resources, as soon as these occur or even at the point where a commitment has been made for them to take place. Accruals based accounting also reinforces the concept of cut-off: Expenditures must be recorded when they are incurred and revenues recorded when they are earned. In summary, accruals based accounting provides more complete information regarding effective commitments. This should assist in the management of EU funds as well as provide external observers with more relevant information. The approach followed by the Court in the audit of the accounts of the European Communities follows standard financial audit methodology. Since this is amply described in the available professional literature, this particular aspect is not covered further in this brochure. In contrast, the audit objective concerning the legality and regularity question presents particular and unusual challenges. These aspects are addressed in some detail in this brochure. 7

10 Main sources for the audit of legality & regularity The Court s methodology is based on two principal sources of audit evidence: An examination of the operation of the supervisory and control systems applied in the collection and disbursement of European funds by Community Institutions, Member States and third countries; these systems are intended to ensure the legality and regularity of revenue and expenditure. Sample checks of transactions relating to revenue and expenditure down to the level of the final beneficiary (also commonly referred to as substantive testing). These principal sources can be complemented by two other sources: An analysis of the Annual Activity Reports and the declarations of the Commission s Directors-General and the procedures in drawing them up. An examination of the work of other auditors, defined as those that are independent of the Community s management and control process. Relation between the different sources Scrutiny of the supervisory and control systems delivers a first level of assurance that errors in transactions are detected and corrected. The extent to which this is true determines the extent of substantive testing which is required to complete the Court s assessment of the legality and regularity of transactions. Complementary evidence can, in some cases, be drawn from the analysis of the annual activity reports and declarations of the Directors-General and the work of other auditors. Other sources of audit evidence may also be taken into consideration. Examples may include: evidence obtained from previous audits; reports resulting from parliamentary enquiries, results of investigations conducted by OLAF etc. Supervisory and control systems Analysis of the supervisory and control systems seeks to establish whether the relevant management procedures are operationally sound. The Court extends its analysis to the supervisory and control systems in the Member States in order to cover delegation risk. This allows the origin of the errors at the level of underlying transactions to be more precisely identified. 8

11 Main sources for the audit of legality & regularity In order to ensure that the budget is correctly implemented, the supervisory and control systems should provide representative information on the implementation and functioning of key controls in respect of their ability to prevent or detect and correct errors and on the residual level of illegal or irregular revenue or expenditure. Without these data, the Commission cannot properly fulfil its obligations in respect of its responsibility for the implementation of the budget. Therefore, the Court s audit of the supervisory and control systems includes also an analysis of the quality of the indicators available. Substantive testing Substantive testing is carried out on the basis of a specific sampling methodology adopted by the Court, derived from the application of established statistical principles. This is based on what is called Monetary Unit Sampling (MUS) with a provision for stratification on the basis of pre-established parameters. The objective is to provide a direct check on the legality and regularity of representative samples of payments, down to the final recipient. Substantive testing may also cover the review of processes and analytical procedures in addition to the examination of the individual transactions. Annual Activity Reports and other auditors reports The Directors-General, in their declarations annexed to the annual activity reports, are required to state, on the basis of sufficient relevant evidence, whether they have reasonable assurance that the systems in place ensure the legality/regularity of the underlying transactions. Additionally, the Commission in its synthesis report on the annual activity reports sets out its general conclusions concerning the legality and regularity of the budget revenue and expenditure. The annual activity reports and declarations of Directors-General and their synthesis are, in auditing terms, management representations and are used as complementary evidence for the evaluation of supervisory and control systems after a detailed review by the Court s auditors. In a similar manner, the work of other auditors can only be used by the Court after an appropriate review of the audit standards that have been applied and of the methodology that has been followed. 9

12 The DAS audit process Diagram 1 The DAS audit process GAIN UNDERSTANDING OF THE AUDIT ENVIRONMENT Collect/update and analyse information about the audit environment: legal framework design and functioning of the supervisory and control systems nature of underlying transactions audit standards to be applied, etc. Determine materiality PLAN AUDIT PROCEDURES Formulate reasonable expectations for: inherent risk quality of supervisory and control systems Plan audit procedures to enable reasonable assurance to be obtained PERFORM AUDIT PROCEDURES AND EVALUATE RESULTS Perform audit procedures: examination of supervisory and control systems substantive testing of underlying transactions analysis of annual activity reports and declarations of Directors-General possibly, work on other sources of evidence (results of other auditors etc.) Evaluate audit results: re-examine/assess preliminary expectations modify, if needed, initial planning on the basis of preliminary audit results perform supplementary audit work, if necessary FORM AUDIT CONCLUSIONS AND AUDIT OPINION Form audit conclusions at the level of the specific assessments Form audit opinion (the DAS) 10

13 The DAS audit process Materiality Diagram 2 Setting materiality As in any other public or private administration, it would be unrealistic in practice to assume that no errors occur. Materiality is a concept that acknowledges that underlying transactions can rarely be absolutely free from all errors, and that a degree of tolerance in their accuracy is therefore acceptable. This concept is also recognised in the international auditing standards. This threshold also serves as an influencing factor both in the calculation of sample sizes for substantive testing and in the interpretation of the audit results achieved (see diagram 2). MATERIALITY Errors are material if they exceed a certain level of error that is above what would be considered to be tolerable. QUANTITATIVE ASPECTS VALUE JUDGEMENT ON MATERIALITY Judging the maximum level of illegal or irregular revenue or expenditure that is acceptable Materiality threshold for DAS audits is set at 2% of total revenue and expenditure QUALITATIVE ASPECTS NATURE CONTEXT In the context of the legality and regularity of underlying transactions, errors are material if they would reasonably affect the decisions of the stakeholders of the audit opinion. In DAS audits, determining materiality in terms of value involves a judgement as to the maximum level of illegal or irregular revenue or expenditure that is tolerable. In general, the materiality threshold for DAS audits is set at 2% of total expenditure or of total revenue of the EU budget for the audit opinion at the level of the DAS. However, the Court may decide to set a different level of materiality or to consider some differentiation of materiality thresholds between budgetary areas to take into account the DAS addressees requirements. 11

14 The DAS audit process Risk analysis The audit cycle A detailed risk analysis is carried out at the planning stage. For each area, the risk analysis determines: the reasonable expectations about the potential level of errors as regards the legality and regularity of the underlying transactions in the area concerned; and The DAS approach follows the conventional audit cycle with due consideration given to all the essential steps, namely: understanding the audit environment; planning the audit; performing the audit and evaluating the results; forming the audit conclusions and audit opinion. the expected distribution of audit work between the two sources, evaluation of supervisory and control systems and substantive testing, with a view to the confidence level considered necessary in order to support the audit conclusion, i.e. the better the systems, the smaller the resulting sample of underlying transactions to be checked. Risk analysis is particularly important since the DAS work on supervisory and control systems is largely concerned with assessing control risk. The areas where inherent risk is highest are those where assurance is needed that controls are managing the relevant risk effectively. Therefore, this must be the main focus of the audit of supervisory and control systems. These are represented graphically in diagram 3. Reasonable assurance The DAS audit approach is designed to provide the Court with reasonable assurance as to whether the underlying transactions, taken as a whole, are free, in all material respects, from errors as regards legality and regularity. In this context, reasonable assurance is a concept relating to the accumulation of audit evidence necessary to draw adequate audit conclusions. This is done at the level of the specific assessments, on which the overall audit opinion (the DAS) is based. In fact, an auditor cannot obtain absolute assurance because there are inherent limitations in an audit that affect the auditor s ability to detect all material errors. Reasonable assurance is obtained when the auditor has reduced audit risk to an acceptably low level through the judicious design and careful implementation of appropriate audit procedures. The relevant audit process is explained in the sections which follow. 12

15 The DAS audit process Confidence level Diagram 3 The DAS audit cycle The assurance obtained from audits of the supervisory and control systems and for tests of transactions should cumulatively allow an acceptable overall confidence level to be reached. This acceptable overall confidence level is the residual value of the audit risk that the auditor is willing to accept. The Court s objective is to reach 95% confidence level as regards the subject matter. Plan audit procedures Audit environment Legal basis 1 Evidence International audit standards 2 Perform audit & evaluate results International auditing standards require that a minimum level of assurance must come from substantive testing, irrespective of the assurance derived from controls or other mitigating factors. On this basis, the DAS assurance model includes substantive testing of a representative sample of transactions, selected randomly from the entire population of the specific area in question. The size of the sample is determined through the use of appropriate statistical methods. In order to control the sampling risk, confidence levels are fixed taking into consideration the inherent risk and control risk identified. Understand the audit environment 1 The DAS 1 IFAC INTOSAI COSO 2 (Art. 248 of the Treaty) Reliability of the accounts Legality & regularity of underlying transactions (Art. 129(4) of Financial Regulation) published with financial statements 2 International Standards of Auditing (ISA) published by IFAC INTOSAI auditing standards COSO Framework Form audit conclusions & audit opinion 13

16 The DAS audit process Assurance model Diagram 4 DAS assurance model The DAS assurance model indicates the level of confidence to be derived from the two principal sources of the DAS. It takes into consideration the particular characteristics of the Court s audit environment. The starting point is the assessment of the inherent risk (high or not high) and the preliminary evaluation of the supervisory and control systems (poor, good or excellent), in order to estimate the degree of confidence that can be derived thereof. On the basis of the results achieved, the extent of substantive testing required to provide the remaining confidence level is determined (see diagram 4). Depending on the assessment of the inherent risk and the evaluation of the supervisory and control systems, assurance from substantive testing is required at one of three levels: If the auditor classifies supervisory and control systems as excellent and assesses the inherent risk as not high, a significant proportion of the audit assurance can be drawn from systems work (high controls assurance) and sample sizes may be reduced. In such a case, minimum substantive testing has to be performed where only a low confidence level needs to be obtained from this source. Assessment of inherent risk high not high Corresponding degree of controls assurance Evaluation of supervisory & control systems poor good excellent Required overall assurance for audit conclusions and audit opinion (DAS) Corresponding degree of substantive assurance Substantive testing focussed standard minimum If the auditor considers the inherent risk as being high and supervisory and control systems as being excellent or good, or the inherent risk as being not high and the supervisory and control systems as being good, the auditor has to perform standard substantive testing. In this case, some assurance can be drawn from the evaluation of supervisory and control systems (medium controls assurance), while the aim is to obtain the major part of the assurance by carrying out substantive testing. If the auditor cannot rely sufficiently on supervisory and control systems to prevent or to detect and correct potential errors (low controls assurance), focused substantive testing has to be performed which aims to provide a high confidence level (more than 90%) by means of increased sample sizes. The audit of the sample not only provides direct evidence on the underlying transactions, but also useful information on any weaknesses of supervisory and control systems that are at the origin of the errors found. 14

17 Performing audit procedures Examination of supervisory and control systems On the basis of the supervisory and control systems, the Commission should obtain the information it needs to ensure that the objectives are being achieved in line with the relevant legal, regulatory and/or contractual requirements. This information should then enable the Commission to assume the responsibility for guaranteeing the implementation of the budget. Supervisory and control systems exist at various levels of management. Some aspects are common, regardless of whether the Commission s management is direct (Commission - beneficiary) or indirect (Commission - Member State - beneficiary). Others are not only specific to each type of management but also to the policy areas and/or measures concerned. All third parties (whether public or private) whose audit and control activities are expressly envisaged by EU legislation must be considered as an integral part of the EU system. The auditor must determine the confidence level to be derived from the evaluation of the supervisory and control systems, together with the attendant risk analysis. This must be supported by analyses of their performance in practice and their actual capacity to guarantee the legality and regularity of the transactions. For each policy area, a programme of tests of control is established in order to evaluate the functioning of the supervisory and control systems according to the five key control objectives to be covered (see diagram 5). Diagram 5 - Key control objectives KEY CONTROL OBJECTIVES Reality and measurement Eligibility of underlying transactions Compliance with other regulatory requirements Correctness of calculations Completeness and accuracy of accounting Substantive testing SUPERVISORY & CONTROL SYSTEMS SHOULD ENSURE THAT: Underlying operations exist and are accurately determined Various Community eligibility criteria are met for the underlying transactions Other (non-eligibility) criteria are met All calculations are correctly undertaken All transactions are accounted for, are not included more than once, are recorded in correct accounting period and at correct value The aim of substantive testing is to obtain direct evidence upon which to formulate the audit conclusion regarding the legality and regularity of the underlying transactions. Substantive testing makes it possible to: IFAC TERMINOLOGY Occurrence / Accuracy Legality & Regularity Legality & Regularity Accuracy Completeness / Accuracy / Annuality assess the legality and regularity of the underlying transactions down to final beneficiary level for the policy area as a whole; provide information (order of magnitude) on the origin, nature, frequency and impact of errors in the population; 15

18 Performing audit procedures identify areas where action needs to be taken so as to avoid further errors; facilitate communication with auditees by producing concrete examples of the shortcomings observed; In general, payment conditions are concerned when elementary legal requirements of the respective aid scheme or contract are affected. On the other hand, other compliance issues are specific conditions to be fulfilled, which result from related policy objectives and related legal obligations or systems requirements. confirm or identify certain weaknesses in the supervisory and control systems which should have revealed the errors encountered. The treatment of errors in substantive testing An important element within substantive testing is the treatment of errors detected. Errors refer to transactions (or parts thereof) and/or actions linked to them which have not been carried out in accordance with the applicable legal and regulatory provisions. One of the pertinent considerations is how to interpret their relative gravity and how to use them in extrapolating the results obtained from the audit sample. In order to determine whether and to what extent the errors detected are relevant for the Statement of Assurance, the Court uses a decision tree approach to establish whether the condition for payment or another compliance issue was affected: Furthermore, the Court classifies errors as being either quantifiable or otherwise non-quantifiable. The former category (quantifiable errors) refers to errors which have a direct and measurable financial impact on the amount of the underlying transactions financed from the Community budget and which can, generally, be readily quantified. The latter category (non-quantifiable errors) describes errors which either do not have a direct financial impact on the amount of an underlying transaction or else where such an impact is very difficult to establish. The most frequent examples of both types of errors are shown hereafter. The so called non-quantifiable errors are not taken into account when extrapolating the sample results to estimate the error for the population as a whole. However, the frequency of non-quantifiable errors detected in the sample will be applied to the entire population. 16

19 Performing audit procedures Most frequent quantifiable errors as regards conditions for payment (with examples) Most frequent non-quantifiable errors resulting from other compliance issues (with examples) Over-declaration Declaration of expenditure not incurred Over-declaration of arable land Declaration of fictive olive oil production Declaration of incorrect expenditure Exceeding of the maximum value of subsidy Non-compliance with contractual conditions Non-compliance with eligibility criteria Expenditure ineligible by nature / outside the eligible period Project ineligible by nature Beneficiary ineligible by nature Lack of audit trail / absence of essential justifying documentation No evidence for co-financing No evidence for expenditure incurred / activities carried out Insufficient documentation of time recording No evidence for calculation of overheads Non-communication of modified state aid rules Non-compliance with tendering rules or other legal obligations having no impact on eligibility Non-respect of deadlines Delay in transfer of funds Delay in presentation of documents Delay in amendment of contracts or decisions Deficiencies which initially were substantial but could be eliminated based on Court s audit results Inconsistent documentation / information Incomplete information / documentation Information / documentation not updated Problems concerning non-essential documentation / information 17

20 Performing audit procedures Diagram 6 - Choosing effective balance between controls and substantive assurance Assessment of inherent risk and evaluation of supervisory & control systems Tendencies concerning level of inherent risk/quality of supervisory & control systems Substantive testing of underlying transactions Auditor cannot rely (sufficiently) on supervisory & control systems High inherent risk / poor supervisory & control systems Focussed substantive testing Auditor can draw a more than insignificant part of the assurance from the evaluation of supervisory & control systems Diminishing inherent risk / improving quality of supervisory & control systems Standard substantive testing Auditor can draw a very high part of the assurance from the evaluation of supervisory & control systems Low inherent risk / excellent supervisory & control systems Minimum substantive testing 18

21 Performing audit procedures Analysis of Annual Activity Reports & declarations by the Directors-General The examination of annual activity reports and declarations of the Directors-General is intended to assess whether they present a fair picture (i.e. with no significant distortions) of the effectiveness of the procedures put in place. In particular, whether supervisory and control systems generate the necessary guarantees insofar as the legality and regularity of the underlying operations are concerned. In this context, two aspects are important: All significant weaknesses should result in a reservation; The reservations should not remove the substance of the declaration. The Directors-General have to base their assessment of the effectiveness of the internal control system on a suitable internal control framework. The Court also reviews whether the reservations presented in the declarations comply with the following criteria: Brief description of the corrective measures foreseen in order to correct the identified weakness and an indication of the expected timetable. In assessing the validity and accuracy of the Directors-General s declarations, the Court s auditors consider the results of their audits on the activities covered by the annual activity reports. The approach developed for the audit of the annual activity reports and declarations of Directors-General are applied by analogy for the Commission s Synthesis Report. Other auditors work In the specific context of the DAS, other auditors means any auditor who gives a professional opinion about expenditure financed by the EU budget but does not do so as part of the internal control arrangements for the management of EU funds. In general, falling under this definition of other auditors will be the national audit institutions in the Member States. In contrast, auditors whose audit opinions are an integral part of the supervisory and control systems (e.g. auditors who certify the correctness of claims for expenditure from the EU budget) cannot be considered under the designation other auditors since they are part of the supervisory and control systems and are to be treated under that heading. Detailed description of their cause and impact on the declaration; Brief assessment of the effect, whether real or potential; 19

22 Performing audit procedures Dealing with differences in the findings obtained The findings concerning the supervisory and control systems might differ from those of the substantive tests. If there is no satisfactory explanation which allows audit conclusions to be drawn in the area concerned, it may be necessary to revise the reasonable expectations expressed during the planning phase and to carry out additional work. See diagram 7. Diagram 7 Treatment of differences in findings obtained Favorable assessment of systems, but significant errors in transactions Unqualified audit opinion impossible Unfavorable assessment of systems, but no errors in transactions Unqualified audit opinion (possible reservations) Re-assessment of systems on the basis of errors found Audit plan may be adjusted Additional substantive procedures 20

23 Forming audit conclusions & an audit opinion Specific assessments In forming the audit opinion, the auditor considers and evaluates the results of the audit procedures, and sets them in the context of the materiality for the audit. The auditor also needs to have regard to wider considerations such as the scope of the audit and whether any limitations have been placed upon it. Whenever a qualitative evaluation is necessary, a high degree of professional judgement is necessary. By applying the principles described above, each specific assessment provides an overall conclusion regarding the legality and regularity of the underlying transactions in the area concerned. The specific assessments also provide an assessment of the performance of the supervisory and control systems at the various levels of management down to final beneficiary level. Diagram 8 Forming audit conclusions and an audit opinion Analysis of annual activity reports and declarations Evaluation of supervisory & control systems Audit conclusions Specific assessments Substantive testing Professional judgement and materiality Qualitative evaluation of results on work on systems Quantitative evaluation of results of substantive testing Analysis of coherence of audit results Audit conclusions Specific assessments Audit opinion The DAS Examination of work of other auditors Audit conclusions Specific assessments If the Court is able to conclude that, on the whole, the accounts give a true and fair view and that the underlying transactions are legal and regular then it can express what is referred to as an unqualified opinion. Where this is not the case, then the opinion must be qualified. Statement of Assurance The specific assessments are consolidated in the form of an overall opinion about the legality and regularity of transactions. Together with the Court s opinion on the reliability of the accounts it forms the Court s statement of assurance. The attached textboxes reproduce some key excerpts from the Court s Statement of Assurance, taken from the latest published Annual Report by the Court. 21

24 Forming audit conclusions & an audit opinion Improvements in systems Diagrams 9 and 10 illustrate the application of the Assurance Model and show how this model is able to cater for changes in the audit environment, in particular the enhancement of supervisory and control systems. As supervisory and control systems improve, more assurance can be drawn from them and the level of substantive testing may be reduced accordingly. 22

25 Forming audit conclusions & an audit opinion Diagram 9 - The DAS decision tree (PRELIMINARY) ASSESSMENT OF INHERENT RISK: HIGH? YES NO (PRELIMINARY) ASSESSMENT OF INHERENT RISK: NOT HIGH? YES (Preliminary) overall evaluation of systems: Poor? NO (Preliminary) overall evaluation of systems: Poor? NO YES (Preliminary) overall evaluation of systems: Good? NO (Preliminary) overall evaluation of systems: Excellent? YES (Preliminary) overall evaluation of systems: Good? NO (Preliminary) overall evaluation of systems: Excellent? YES YES YES YES Low controls assurance (Confidence level: 0%) Medium controls assurance (Confidence level: 15%) Medium controls assurance (Confidence level: 28%) Low controls assurance (Confidence level: 3%) Medium controls assurance (Confidence level: 28%) High controls assurance (Confidence level: 50%) Focussed substantive testing Confidence level: 95% Standard substantive testing Confidence level: 80% Standard substantive testing Confidence level: 67% Focussed substantive testing Confidence level: 92% Standard substantive testing Confidence level: 67% Minimum substantive testing Confidence level: 45% Diagram 10 The different scenarios for DAS work Assessment of inherent risk Not high High Overall evaluation of supervisory and control systems Excellent Good Poor Excellent Good Poor Assurance obtained from combined risk assessment High controls assurance Medium controls assurance Low controls assurance Medium controls assurance Medium controls assurance Low controls assurance Residual level of substantive testing to be carried out Minimum substantive testing Standard substantive testing Focussed substantive testing Standard substantive testing Standard substantive testing Focussed substantive testing Minimum degree of confidence to be derived from substantive testing (%)

26 Glossary Accounting system Accrual based accounting Activity Based Budgeting Activity Based Management (ABM) Annual activity reports and declarations of Directors- General Audit Audit conclusion Audit objective Audit opinion Audit procedure An accounting system is the series of tasks and records of an entity by which transactions and events are processed as means of maintaining financial records. Such systems identify, assemble, analyse, calculate, classify, record, summarise and report transactions and other events. Accruals based accounting is an accounting method of recording transactions through which revenues are recorded when earned and expenditures are recorded when incurred, whether or not the transactions have been finally settled by a receipt or a disbursement of cash. Activity Based Budgeting is budgeting and financial management in an integrated manner on the basis of activities closely linked to the Commission s priorities and objectives. Activity Based Management is an integrated management methodology covering all aspects of the Commission s work, including objective prioritisation, planning and programming, budgeting, management and reporting methods. Each Director-General or Head of Service shall issue an annual activity report which shall address the DG/Service environment, the main achievements, the management and control issues (i.e. internal control), and include the annual accounts. In his/her capacity of Authorising Officer by delegation, he/she shall also sign a declaration in which he/she states, on the basis of the facts in his/her possession, that he/she has reasonable assurance that, in particular, the control procedures put in place give the necessary guarantees concerning the legality and regularity of the underlying transactions. The declaration may contain reservations designed to highlight risks associated with the operations and actions managed by the Directorate-General or Service and, where appropriate, an indication of the remedial measures taken or planned by the Authorising Officer by Delegation. The objective of an audit is to enable the auditor to express an independent opinion on whether the financial statements of an organisation are prepared, in all material respects, in accordance with an identified reporting framework or on whether an activity carried out by an organisation has been performed in accordance with a set of predetermined criteria. The audit conclusions are the overall assessment of the audit findings for a defined area. In forming the audit conclusions, the auditor must consider and evaluate the results of the audit procedures and set them in the context of the materiality of the audit. In the framework of the DAS, each specific assessment must include an overall conclusion regarding the legality and regularity of the underlying transactions in the area concerned. At a general level the audit objectives are defined by reference to the nature of the audit (financial audit or audit of sound financial management). Financial audits of revenue and expenditure have the general audit objectives of legality and regularity; completeness, reality of operations, measurement and presentation and publication. For a balance sheet audit, existence and ownership replaces reality of operations and valuation replaces measurement. Audits of sound financial management have the general audit objectives of economy, efficiency and effectiveness. Within each audit task, it is necessary to define, as part of the planning process, the specific audit objectives of the task. The audit opinion is a clear written expression of opinion on the financial statements or the legality and regularity of underlying transactions as a whole. The two main types of opinions are unqualified opinion and qualified opinion. An audit procedure is the method used to obtain the necessary audit evidence and to analyse it. For tests of control and substantive tests, there are four types of procedures that the auditor can use. These are inspection, observation, inquiry and confirmation and computation. Analytical procedures are mainly used at the planning stage and at the end of the audit. 24

27 Glossary Audit risk Audit scope Centralised management Condition for payment Confidence level Control environment Control risk DAS Decentralised management Delegation risk Disclosure Error European Anti- Fraud Office (OLAF) Final beneficiary Audit risk is the risk that the auditor expresses an inappropriate audit opinion. Audit risk can be broken down in three components: inherent risk, control risk and detection risk. The audit scope refers to the whole area on which the audit procedures deemed necessary in the circumstances (type of audit and of report foreseen, specific objectives and constraints, etc...) to achieve the objectives of the audit will be applied. When the Commission implements the budget in the form of centralised management, the various tasks are performed either directly by Commission departments or indirectly by executive agencies, traditional agencies or national public-sector bodies or bodies governed by private law with a public service mission (so-called national agencies ). Conditions for payment are in general the elementary legal requirements of the respective aid scheme or contract (e.g. respect of eligibility rules or of obligations concerning recoveries). For practical reasons, the assessment can be based on the following question: Should the amount paid have been different if the error had been known at the moment of the payment authorisation? The confidence level (or assurance level) is the converse of audit risk. The higher the degree of confidence desired, the more audit testing must be done. The policy of the Court for all financial audits is to reach a confidence level of 95%. Thus an audit risk of 5% is accepted. The control environment means the overall attitude, awareness and actions of senior and line management regarding internal control and its importance within the entity. Control risk is the risk that internal control procedures will fail to prevent or detect and correct on a timely basis material errors or material failures in financial management. Such a failure may arise either because of the absence of appropriate control procedures or because existing internal control procedures do not operate effectively, continuously and consistently. Since 1994 the Court has been required by the Treaty to provide a Statement of Assurance (DAS - French acronym for "déclaration d assurance"), comprising an opinion on the reliability of the European Union s financial statements, and an opinion on the legality and regularity of the transactions underlying them. When the Commission implements the budget by decentralised management, the different tasks are delegated to third countries in accordance with the provisions of the Financial Regulation. While the European Commission is responsible for implementation of the EU budget (Art. 274 of the Treaty), a major part of the EU budget is managed by Member States (and non-member States given partnership arrangements) in the framework of shared and joint management forms. This separation between authorisation, responsibility and implementation gives rise to the "delegation risk". The term disclosure refers to the presentation of certain information (usually in the financial statements and related notes or in the annual activity reports and their synthesis). The disclosure requirements for the various Community bodies are set out in the bodies Financial Regulations and Implementing Rules (or equivalent) or determined by the needs of the users of the relevant information. Transactions (or parts thereof) and/ or actions linked to them which have not been carried out in accordance with the legal and regulatory provisions applicable. The Anti-Fraud Office, OLAF, has been established by Commission Decision (1999/352/EC, ECSC, Euratom of 28 April 1999, OJ L 136/20). Its overall responsibility extends beyond the protection of financial interests of the Community to include all activities relating to the need to safeguard Community interests against irregular conduct liable to give rise to administrative proceedings. The last body or person in the chain to receive payment from the EU budget. 25

28 Glossary Inherent risk Joint management Key controls Legality and regularity Management representations Materiality Other compliance issue Qualified audit opinion Representativeness Reservation Sampling Sampling risk Inherent risk is the risk related to the nature of the activities, operations and management structures, that errors or failures in financial management will occur which, if not prevented or detected and corrected by internal control procedures, will cause the accounts to be unreliable, the underlying transactions to be materially illegal or irregular or financial management to be unsound. Joint management is a specific form of management by international organisations, to which the Commission can have recourse if a number of conditions are met (notably the pooling of resources from a number of donors), where it is not reasonably possible or appropriate to assign the share contributed by each donor to each type of expenditure. In the context of the DAS, key controls are those control procedures - identified by the auditor during the evaluation and assessment of supervisory and control systems -, which are essential to ensure legality and regularity of the underlying transactions. The financial audit objective of legality and regularity seeks to ensure that a transaction conforms to the applicable laws and regulations and is covered by sufficient budgetary appropriations. Representations made by management to the auditor during the course of an audit, either unsolicited or in response to specific inquiries. Materiality is an expression of the relative significance or importance of an item or of a group of related items. An item or group of items is considered to be material when it is so significant as to merit being drawn to the attention of the readers of an audit report or opinion of the Court. An item or group of items may be material because of its value, because of its nature or because of the context in which it occurs. Other compliance issues are specific conditions to be fulfilled, which result from related policy objectives and legal obligations or systems requirements (e.g. compliance with the "good agricultural and environmental conditions" (GAEC) in the field of Agriculture, or infringements relating to publicity or system requirements in the area of Structural Actions). The non-compliance with these requirements does not mean that the condition for payment is affected but it implies a financial risk and/or could lead to financial corrections to be borne by Member States or fines to be paid by final recipients of grants. The auditor s report contains a clear written expression of opinion on the financial statements or the legality and regularity of underlying transactions as a whole. A qualified opinion is expressed when the auditor concludes that an unqualified opinion cannot be expressed but that the effect of any disagreement with management, or limitation on scope is not as material and pervasive as to require an adverse opinion or a disclaimer of opinion. When performing substantive testing, the auditor uses sampling techniques to draw conclusions about the population from which the sample is drawn. For these conclusions to be valid, the sample must be representative of the population. A reservation is expressed, when it is concluded that an unqualified opinion is not possible but that the effect of errors or systems weaknesses is not as material and pervasive as to require an adverse opinion. Audit sampling (sampling) involves the application of audit procedures to less than 100% of items within a class of transactions or account balance such that all sampling units have a chance of selection. This will enable the auditor to obtain and evaluate audit evidence about some characteristic of the items selected in order to form or assist in forming a conclusion concerning the population from which the sample is drawn. Audit sampling can use either a statistical or a non-statistical approach. Sampling risk arises from the possibility that the auditor s conclusion, based on a sample selected using a statistical or non-statistical approach, may be different from the conclusion that would be reached if the entire population were subjected to the same audit procedure. 26