Auditing ERP systems without specific CAATs

BRAZILIAN COURT OF AUDIT Auditing ERP systems without specific CAATs 21 st Meeting WGITA Kuala Lumpur, Jan, 2012

Auditing ERP Systems without specific CAATS Agenda Brazil and IT Audit Secretariat background Audit opportunities and risks Survey on ERP systems in the Brazilian Federal Public Administration Benchmarking of audit methodologies Audit methodology Conclusion

Brazil background Country data 5 th largest country in the world 6 th GDP in the world area: 8,500,000 sq. km (2.5 x The European Community) population: 190,000,000 inhabitants 84 th HDI Democratic Federative Republic Brazilian Court of Audit (TCU) Federal level 3

IT Audit Secretariat background Created in August 2006 to undertake audits that require specialized knowledge in IT to research, develop and disseminate methods on IT audit to elaborate and provide IT audit training 4

IT Audit Secretariat background Sefti s Role Business: External auditing of information technology governance in the federal government. Mission: To ensure that information technology adds value to the business of the federal government for the benefit of society. Vision: To be a unit that achieves excellence in improving and auditing information technology governance. 5

Auditing ERP Systems without specific CAATS Brazil and IT Audit Secretariat background Audit opportunities and risks Survey on ERP systems in the Brazilian Federal Public Administration Benchmarking of audit methodologies Audit methodology Conclusion 6

Audit opportunities Court Decision All of the national energy areas are supported mainly by ERP systems Company #1 (SOX Compliance) revenues in 2010: US$ 118,3 bi Company #2 (SOX Compliance) revenues in 2010: US$ 15,2 bi 7

Audit risks Lack of knowledge of auditors regarding the topic No prior audits on the topic carried out by TCU Lack of a support tool (CAATs) to audit controls related to the application of ERP systems 8

Survey 57 national public companies Most in the energy business (Petroleum and Electricity) 49% of them use ERP systems and 33% plan on using ERP systems in the medium term Respondents by category 18% Use 49% Plan 33% Don t use 10

Survey 3 main suppliers SAP is the leader, followed by Totvs (a national company) and by Oracle Supplier Quantitative Distribution 25% SAP 36% Totvs 14% Oracle Others 25% 11

Survey Cost of acquisition of licenses and customization approximately US$ 666 million Scope of benefits from implementation of ERP system Benefits Categories Information Security Work process Management issues Controls Financial Others 0% 20% 40% 60% 80% 100% 12

Benchmarking (Experientia Mutua Omnibus Prodest) INTOSAI Readings IntoIT Issue 27, December 2008 Assuring SAP (Australia) IntoIT Issue 28, April 2009 Visits Dutch Experiences with ERP Systems Country Focus South Africa 19th Meeting of Intosai Working Group for IT Audit (WGITA) SAP in public administration (Netherlands) RMAS (Risk Management & Audit Services) at Harvard University ANAO (Australian National Audit Office) SAP Assure software 14

Audit methodology Five companies selected Company #1 - (SOX Compliance) revenues in 2010: US$ 44,4 bi Company #2 (SOX Compliance) revenues in 2010: US$ 15,2 bi Company #3 - revenues in 2010: US$ 7 bi Company #4 - (SOX Compliance) revenues in 2010: US$ 3 bi Company #5 - revenues in 2010: US$ 1,1 bi 16

Audit methodology Audit Scope Focus on evaluation of general controls, due to the lack of a support tool for evaluating application controls Use of globally accepted audit criteria (Cobit 4.1, ISO 27.002, ISO 31.000, ISO 15.999) and national legislation 10 audit questions associated to 49 possible findings Survey with 9,000 users from the selected companies 17

Dimensions MANAGEMENT OF ERP SYSTEM AND IT PLANNING PROCESSES AND METHODS OF SUPPORT PERFORMANCE OF THE INTERNAL AUDIT CONTRACTS AND LEGAL ASPECTS INFORMATION SECURITY CONTROLS USER SATISFACTION APPLICATION CONTROLS ACQUISITION MODULE Audit questions Q1. Is management of the ERP system based on IT plans and policies? Q2. Is a cost-benefit analysis of the investments in the ERP system carried out? Q3. Do the professionals who support and use the ERP system undergo appropriate training and receive information that is appropriate to carry out their activities? Q4. Does the IT area count on processes and methods to support the ERP system? Q5. Are the management and use of the ERP system overseen by internal audit? Q6. Do the contracts related to the ERP system meet the legal provisions? Q7. Have the general IT controls associated with the security of the ERP system been implemented according to best practices? Q8. Have the controls of access to the ERP system been implemented according to best practices? Q9. Are users satisfied with the ERP system? Q10. Have the existing controls in the ERP system for making public acquisitions been implemented according to legislation and to best practices?

Findings Q9: User satisfaction Length of time using system Did not respond 0% Less than 1 year 3% Between 1 and 3 years 12% More than 5 years 56% Between 3 and 5 years 29% 19

Findings Q9: User satisfaction Distribution of length of time using system 5% 24% Use the ERP system more than other systems Use other systems more than ERP system 42% Use ERP and other systems for almost the same time 29% Did not respond 20

Findings Q9: User satisfaction Influence of system use 4% 0% 9% Increases my productivity 14% Does not influence my productivity Decreases my produtivity 73% I don t know Did not respond 21

Findings Q9: User satisfaction Need to reenter ERP system information in other systems 1% Yes 61% 38% No Did not respond Need to reenter other systems information in ERP system 1% Yes 64% 35% No Did not respond 22

Findings Q9: User satisfaction General level of satisfaction with system use 8% 0% 33% 12% 47% Totally satisfied Very satisfied Partially satisfied Dissatisfied Did not respond Aspects of dissatisfaction with system Did not respond 22% Other 26% The system is not trustworthy 2% The system is frequently offline 3% The system does not have the operations I need 11% The system is slow 11% The system is difficult to use 25% 23

Conclusion It is possible to audit ERP systems without the use of specific CAATs The steps suggested are: Carrying out a survey on the status of ERP use in the country Benchmarking of audit methodologies Carrying out survey among users of the systems of chosen companies Creating and executing a methodology for evaluating general controls mainly 25

Conclusion If the SAI does not have previous experience or resources to acquire specific CAATs to help in ERP system audit, it should invest in knowledge and motivation in order to face the challenges of a task of such importance 26

Thank You! sefti@tcu.gov.br 55 (61) 3316-5371 www.tcu.gov.br/fiscalizacaoti 27