IT Support through CAATTs - Systematic Requirements Analysis and Design for Process Audit



Similar documents
Continuous Auditing in Big Data Computing Environments: Towards an Integrated Audit Approach by Using CAATTs

Continuous Auditing in Big Data Computing Environments: Towards an Integrated Audit Approach by Using CAATTs

NASCIO EA Development Tool-Kit Solution Architecture. Version 3.0

The Relationships between Computer Auditing Activity and. Performance

Risk Knowledge Capture in the Riskit Method

Partnering for Project Success: Project Manager and Business Analyst Collaboration

2. Auditing Objective and Structure What Is Auditing?

THE ROLE OF KNOWLEDGE MANAGEMENT SYSTEM IN SCHOOL: PERCEPTION OF APPLICATIONS AND BENEFITS

Internal Audit Quality Assessment. Presented To: World Intellectual Property Organization

JOURNAL OF OBJECT TECHNOLOGY

Ten steps to better requirements management.

Continuous auditing: the audit of the future

A Study on RE Process Models for Offshore Software Development

U S I N G D A T A A N A L Y S I S T O M E E T T H E R E Q U I R E M E N T S O F R I S K B A S E D A U D I T I N G S T A N D A R D S

THEORETICAL APPROACHES TO EMPLOYEE APPRAISAL METHODS

Quality Ensuring Development of Software Processes

BUSINESS STRATEGY SYLLABUS

Methods Commission CLUB DE LA SECURITE DE L INFORMATION FRANÇAIS. 30, rue Pierre Semard, PARIS

Module 2 IS Assurance Services

4 Testing General and Automated Controls

Practice Overview. REQUIREMENTS DEFINITION Issue Date: <mm/dd/yyyy> Revision Date: <mm/dd/yyyy>

Process-Based Business Transformation. Todd Lohr, Practice Director

Competence Requirements for Audit Professionals

Chapter 3 Chapter 3 Service-Oriented Computing and SOA Lecture Note

Learning Outcomes Implementation Guidance - Revised Staff Questions & Answers Document

INTERNATIONAL FRAMEWORK FOR ASSURANCE ENGAGEMENTS CONTENTS

Reaching CMM Levels 2 and 3 with the Rational Unified Process

QUAๆASSURANCE IN FINANCIAL AUDITING

CDC UNIFIED PROCESS PRACTICES GUIDE

Business Analysis Standardization & Maturity

An Investigation of Factors Affecting Marketing Information Systems Use

In recent years, information technology (IT) used by firms,

V&V and QA throughout the M&S Life Cycle

Factors Influencing Audit Technology Acceptance by Audit Firms: A New I-TOE Adoption Framework

Towards Collaborative Requirements Engineering Tool for ERP product customization

Maintaining the Relevance of the Uniform CPA Examination

The objective of Software Engineering (SE) is to build high quality software. within a given time and with a predetermined budget (Sommerville, 2007).

Using Requirements Traceability Links At Runtime A Position Paper

The Accounting Information Systems Curriculum: Compliance with IFAC Requirements

Safety Regulation Group SAFETY MANAGEMENT SYSTEMS GUIDANCE TO ORGANISATIONS. April

Business Information Systems. IT Enabled Services And Emerging Technologies. Chapter 4: Facilitated e-learning Part 1 of 2 CA M S Mehta, FCA

Interview studies. 1 Introduction Applications of interview study designs Outline of the design... 3

Towards Continuous Information Security Audit

The Impact of Enterprise Resource Planning (ERP) System on the Cost and Price of Auditing Auditor s Perspective

Quick Guide: Meeting ISO Requirements for Asset Management

Using Business Process Simulation to Assess the Effect of Business Rules Automation

INTERNATIONAL STANDARD ON ASSURANCE ENGAGEMENTS 3000 ASSURANCE ENGAGEMENTS OTHER THAN AUDITS OR REVIEWS OF HISTORICAL FINANCIAL INFORMATION CONTENTS

ONTARIO'S DRINKING WATER QUALITY MANAGEMENT STANDARD

Certified Information Professional 2016 Update Outline

INTERNAL AUDITING S ROLE IN SECTIONS 302 AND 404

Revealing the Big Picture Using Business Process Management

Establishing a Quality Assurance and Improvement Program

Secondary Data Analysis: A Method of which the Time Has Come

Challenges in Developing a Small Business Tax payer Burden Model

A Model for Effective Asset Re-use in Software Projects

School of Advanced Studies Doctor Of Management In Organizational Leadership/information Systems And Technology. DM/IST 004 Requirements

CDC UNIFIED PROCESS PRACTICES GUIDE

PROJECT MANAGEMENT PLAN TEMPLATE < PROJECT NAME >

Fourth generation techniques (4GT)

City University of Hong Kong. Information on a Course offered by Department of Information Systems with effect from Semester B in 2013 / 2014

A Comparison of SOA Methodologies Analysis & Design Phases

Navigating the Standards for Information Technology Controls

EVOLVING THE PROJECT MANAGEMENT OFFICE: A COMPETENCY CONTINUUM

INFORMATION SYSTEM AUDITING AND ASSURANCE

Professional Development for Engagement Partners Responsible for Audits of Financial Statements (Revised)

Background. Audit Quality and Public Interest vs. Cost

Improving Traceability of Requirements Through Qualitative Data Analysis

How To Write An Impactful Audit Report

An Instructional Design for Data Warehousing: Using Design Science Research and Project-based Learning

Qlik UKI Consulting Services Catalogue

ERP implementation and Organization Changes

The Impact of Service Oriented Architecture (SOA) on IT Auditing

Social Team Characteristics and Architectural Decisions: a Goal-oriented Approach

School of Advanced Studies Doctor Of Management In Organizational Leadership. DM 004 Requirements

FFIEC Cybersecurity Assessment Tool

Frequently Asked Questions in Identifying and Assessing Prospective Risks

CHEA. Accreditation and Accountability: A CHEA Special Report. CHEA Institute for Research and Study of Acceditation and Quality Assurance

STRATEGIC DECISION-MAKING IN A PROFESSIONAL SERVICE FIRM

Implementing the International Standards for Supreme Audit Institutions (ISSAIs): Strategic considerations

QUALITATIVE RESEARCH. [Adapted from a presentation by Jan Anderson, University of Teesside, UK]

Investigating Role of Service Knowledge Management System in Integration of ITIL V3 and EA

Why Data Mining Research Does Not Contribute to Business?

Insights into Large Audit Firm Sampling Policies

The Framework for Quality Assurance

G11 EFFECT OF PERVASIVE IS CONTROLS

A Variability Viewpoint for Enterprise Software Systems

Comparing Methods to Identify Defect Reports in a Change Management Database

ISO Gap Analysis - Case Study

Overview. The Knowledge Refinery Provides Multiple Benefits:

IBM Information Management

ADOPTION OF OPEN SOURCE AND CONVENTIONAL ERP SOLUTIONS FOR SMALL AND MEDIUM ENTERPRISES IN MANUFACTURING. Mehran G. Nezami Wai M. Cheung Safwat Mansi

Initial Professional Development Technical Competence (Revised)

Fundamentals of Measurements

Sustaining the Benefits of Action Research in Decision Support Tools Development: Lessons from an Urban Water Utility in Africa

Interviews and Focus Groups in Advertising, Public relations and Media

Organization of data warehousing in large service companies - A matrix approach based on data ownership and competence centers

Criticism of Implementation of ITSM & ISO20000 in IT Banking Industry. Presented by: Agus Sutiawan, MIT, CISA, CISM, ITIL, BSMR3

How To Understand The Business Analysis Lifecycle

Project Knowledge Management Based on Social Networks

Transcription:

Systematic Requirements Analysis and Design for Process Audit IT Support through CAATTs - Systematic Requirements Analysis and Design for Process Audit Completed Research Paper Andreas Kiesow University of Osnabrück Andreas.Kiesow@uni-osnabrueck.de Sebastian Bittmann University of Osnabrück Sebastian.Bittmann@uni-osnabrueck.de Oliver Thomas University of Osnabrück Oliver.Thomas@uni-osnabrueck.de Abstract Due to the increasing operation of automated and autonomous Accounting Information Systems (AIS) in the recent decades, the audit of financial statements through process audits has risen in complexity. Additionally, process audits have to be managed in a field of tension because of intensified legal framework and the ever-growing size of data. Therefore, auditors demand for solutions, which support an efficient and high quality examination of financial statements. According to the principles of Design Science, an analysis of essential requirements of the auditors is examined in this work. With respect to the auditor s needs, a specification of process audit is conducted from expert interviews. This sets the baseline for the design of innovative IT artifacts for a guided and systematic support for all activities of process audit. The paper concludes with a critical discussion and highlights the implications for future work. Keywords Process Audit, AIS, Computer-Assisted Audit Tools and Techniques, Requirement Analysis Twentieth Americas Conference on Information Systems, Savannah, 2014 1

Accounting Information Systems Introduction The audit s purpose and main tasks are defined in national and international standards such as AICPA SAS No. 1 or IFAC ISA 200. According to these standards, the objective of financial audits is to gain a valid expression on the fairness and truth of the financial statements, i.e. if they are presented in accordance with legal frameworks and accounting principles (AICPA 1972; IFAC 2013a). Financial statements and the related financial accounting information are produced by Accounting Information Systems (AIS) as well as external reporting systems (Bushman and Smith 2001). Since automatic and autonomous function of AIS increase permanently (Gelinas et al. 2011), financial statements are results of computer-assisted accounting processes. Therefore, reasonable audits of these computer-assisted accounting processes (process audit) are essential to gain a sufficient opinion of a fair and true presentation of the financial statements. Computer-assisted accounting processes are characterized by automatic generation of data and their autonomy of data processing, which leads to a higher and evergrowing size of data (Vasarhelyi et al. 2012). The manual audit of mass data is usually not cost-efficient (Zabihollah Rezaee, Rick Elam 2001), it does not enable the detection of systematic failures through the processing and it does not allow statements about completeness and accuracy of the processed data within the AIS (IFAC 2013b). Therefore, IFAC ISA 315.83 suggests the use of computer-assisted audit tools and techniques (CAATTs). CAATTs exist in manifold types such as integrated check sums or trend analysis. However, due to the rapid digitalization and increasing complexity of accounting processes, the development of innovative CAATTs is gaining in importance. To create a basis for CAATTs support, essential requirements of process audits have to been carved out due to the analysis of process audit activities and the assessment of their individual specifications. These requirements are manifold and need to be analyzed through a detailed investigation. In addition, the conduction of audits as projects has to be considered. On the one hand, annual audits are subject to time and cost restrictions (McDaniel 1990). On the other hand, the attainment of high quality results requires reasonable understanding and sufficient evidences of the audited process (Carlin and Gallegos 2007). Therefore, process audits fit in the definition of the Iron Triangle, which is characterized by a reduction of time and cost with consideration to the need for increasing quality of the results (Atkinson 1999). Through tremendous changes in the legal framework, such as Sarbanes-Oxley Act Section 404 (Sarbanes-Oxley Act 2002) for the US or the sought reform of the audit sector in the European Union (European Commission 2013), efficiency and effectiveness of financial audit have gained in importance (Braun and Davis 2003). The relation between AIS, process audit and the challenges they are facing is shown in Figure 1. Although this area is primarily dominated by the practical considerations of audit companies, the usage of information systems for process audits is a prevalent and dominant requirement which needs a holistic and theoretical exploration. Obviously, practical users can benefit from research work and, vice versa, researchers have to be aware of practical needs. Therefore, the authors of this paper seek to close this gap by identifying and analyzing the relevant and important design directions for future CAATTs in the area of process audits. To do so, interviews of experts from an audit company were conducted and evaluated through a scientific approach. Thereby, the need for appropriate computer-assisted support increases and leads to the following research question: RQ1: What are essential requirements to the successful accomplishment of a process audit? In 2008, a study was published which researched the auditor s acceptance of CAATTs. The initial point of this study was the finding that the auditor s acceptance of CAATTs is not sufficiently pronounced (Janvrin et al. 2008). Hence, this paper attempts to highlight possibilities for the computer-assisted support in the different activities of process audit, considering its needs and restrictions and, thus, to strive to sharpen the auditor s awareness for the usage of CAATTs. Specifically, the different phases of process audits are examined along with their specific potential for computer-assisted support. Thereby, the second research question was derived accordingly. RQ2: Which requirements of process audit can be covered by the development of innovative CAATTs? The paper is structured as follows. First, related work in the fields of process audit and CAATTs is analyzed. Then the research method that is used is presented. In the following section, a general concept of process audit is derived from theoretical background, a generally used standard and expert interviews. 2 Twentieth Americas Conference on Information Systems, Savannah, 2014

Systematic Requirements Analysis and Design for Process Audit This concept is specified in its phases and activities as well as visualized in a graphical representation. With respect to the research questions, essential requirements are carved out along the phases of process audit and analyzed regarding the current support by CAATTs. Further, proposals for potential support by CAATTs are described and discussed. Finally, the paper concludes with an outlook, which emphasizes further work in this field. Volatile Legal Framework Accounting Transaction Accounting Processes Financial Statements Legal Framework Enterprise Assessesment Accounting Information System Control Environment Integration Growing Data Volume Certificate Computer Assisted Audit Tools and Techniques Audit Transaction Level Process Audit Time Efficiency & Effectiveness True and Fair View? Cost Quality Figure 1: Audit of computer-assisted Accounting Processes Related Work and Theoretical Background Reviewing Process Audits The audit of computer-assisted processes was the object of versatile research works and studies. Regarding the auditor s skills in Information Systems (IS), in 1973 the EDPACS Newsletter stated that auditors are required by their professional standards to be systems analysts and need essentially the same background in EDP [Electronic Data Processing, author s note] as the analysts who design the clients' computer-based accounting processes (Samson 1973). In 1990, VASARHELYI AND HALPER developed the Continuous Process Auditing System (CPAS) that is designed to measure and monitor large systems, drawing key metrics and analytics into a workstation environment and combines Process Audit with Continuous Audit (Vasarhelyi and Halper 1991). Due to the development of CPAS, the dependency on cooperation with the client and the need to audit the integrated controls became apparent. However, this approach focuses on the technological aspect for the test of automated controls. It neglects testing organizational controls as well as the documentation of a process understanding. In 1998, BUCHANAN AND GIBB published a review of methods for auditing IT strategy. In their work, they describe the relationship between processes and the underlying IT. They conclude that none of the reviewed methods provide a comprehensive information auditing solution or completely fulfil this strategic role (Buchanan and Gibb 1998). HAMMER proposes a methodology for the performance of business processes (Process and Enterprise Maturity Model, PEMM). However, the focus of this methodology lays primarily on the performance of the process, which is not part of the purpose of the audit (Hammer 2007). Completeness and correctness of data through the processing of data by AIS is neglected. CARLIN AND GALLEGOS note An IT audit examines the control structure of an organization s business processes, which may or may not be entirely computerized, to validate the organization s information assurance practices (Carlin and Gallegos 2007). In their work, they carried out essential needs of the IT Auditor, such as training and knowledge of the audited organization. A substantial survey in the field of process audit was published by SCHULTZ et al. in 2012. Through conducting expert interviews they seek to build a specific modeling language for the auditor s purposes (Schultz et al. 2012). Twentieth Americas Conference on Information Systems, Savannah, 2014 3

Accounting Information Systems Reviewing Computer-Assisted Audit Tools and Techniques The definitions of Computer-assisted Audit Tools and Techniques (CAATTs, also: Computer-assisted Audit Techniques, Computer-aided Audit Tools etc.) vary in different publications and is has changed over time. For instance, the definition of SINGLETON focuses on the gathering of evidences, thus, CAATTs are defined as computer tools and techniques that an auditor (external or internal) uses as part of their audit procedures to process data of audit significance contained in an entity s information systems (Mahzan and Lymer 2008). Considering that the purpose of this paper is eliciting requirements for the developing computer-assisted support for all phases within process audits this definition has to be stretched. In the definition of SAYANA, CAATTs are certain software used by auditors to perform audits and to reach the objectives of the audit activities (Sayana 2003). In the broad definition of BRAUN and DAVIS CAATTs include any use of technology to assist in the completion of an audit (Braun and Davis 2003). According to this definition, the possibilities of computer-assistance in audit are numerous, since word processing programs and spreadsheets can be easily used and adapted by end users. Consequently, the tools range from electronic guidelines, checklists and templates to individual data-processing. The writing of audit reports can be supported by the extensible Business Reporting Language (XBRL, Taylor and Dzuranin 2010). A history of automated control testing is provided by AICPA (Coderre and Police 2005). A recent categorization of CAATTs related to the testing of financial data is spread in Standards such as SAS No. 94 (AICPA 2001) and substantiated by the Information Systems Audit and Control Association (ISACA, Cerullo and Cerullo 2003). ISACA also published the IS Auditing Guideline G3 regarding the use of CAATTs since it serves as an important tool for the IS auditor to evaluate the control environment in an efficient and effective manner (ISACA 2008). In this work, it is assumed that the use of appropriate CAATTs increases efficiency and leads to effective results. Cost of development and producing wrong results are not examined in this work. Research Methodology The authors strive for the development of CAATTs, which support the auditors to face the challenges mentioned in the introduction. Therefore, the applied research approach of this work is Design Science according to HEVNER, in the meaning of the development of appropriate artifacts for human (i.e. auditor s) purposes. Further, HEVNER stated that A design artifact is complete and effective when it satisfies the requirements and constraints of the problem it was meant to solve (Hevner et al. 2004). Therefore, essential requirements for the successful process audit (RQ1) und the development of innovative CAATTs (RQ2) have to be rigorously analyzed. The research approach applied in this paper is shown in Figure 2 and described in the following subsections. Research Questions (1st Section) Research Methodology (3rd Section) Results Theoretical Background (2nd Section) Review of Standard Expert Interviews RQ1: What are essential requirements to the successful accomplishment of process audit? RQ2: Which requirements of process audit can be covered by the development of innovative CAATTs? Reviewing Process Audit Reviewing CAATTs Q1: What are the essential phases of process audit, in your opinion? Q2.1: What requirements are indispensable to the procedure of process audits? Q3.1: Which phases of process audit are currently supported by CAATTs? Q2.2: What are further important prerequisites for the successful accomplishment of process audit? Q3.2: In which phases do you see potential support by CAATTs? Specification of Process Audit (4th Section) Requirement Analysis (5th Section) Potential CAATTs Support (6st Section) Figure 2: Research Approach 4 Twentieth Americas Conference on Information Systems, Savannah, 2014

Systematic Requirements Analysis and Design for Process Audit Requirements Analysis Initial points are the research questions mentioned in the introduction. To gain an understanding of the current state of literature, related work and theoretical background are analyzed (2nd Section). The goal of the 3rd Section is to answer RQ1. Hence, it is to gain an understanding of process audit, which requires the alignment of different resources, which are in their nature differently to each other as Restrictive, Descriptive and Prescriptive, specification (Curtis et al. 1992). These resources are: The audit s purpose as mentioned in the introduction and the legal framework (Restrictive), interviews with three experienced auditors (Descriptive) and professional standards (Prescriptive). To gather information about the auditor s needs, structured requirements analysis has to be carried out in 4th Section (Tiwari et al. 2012). Since the definition of requirements has to capture all aspects of system development prior to actual system design (Ross and Schoman 1977), the analysis is succeeded by the establishment of essential requirements. According to GOGUEN and LINDE, the question of how to figure out what the stakeholders need arises (Goguen and Linde 1993). There are manifold methodologies and techniques to answer this question (Hickey and Davis 2003). Since interviews are efficient at describing the interactions between system and stakeholders (Sabahat et al. 2010), in this paper three expert interviews are conducted with the intent to learn of a collaborative technique (Tiwari et al. 2012). Finally, the requirements are analyzed (5th Section) and discussed (6th Section), which sets a baseline for the answer to RQ2. Sample Selection As mentioned above, an expert interview approach is selected to describe the interaction between system and stakeholders. First, these experts have to be highly aware in the area of process audits, i.e. broad experience of conducting process audits in practice (at least two years of experience and Certified Information Systems Auditor (CISA) exam). Second, the experts have to be aware with already existing CAATTs. At last, some diversity has to be included in the sample regarding industries, internal or external audit perspective and different grades of experience to cover managerial and operational views on process audits. Therefore, the authors see a heterogeneous purposive sample approach according to PATTON as an appropriate method to select a sample, which represents a type in relation to the key criterions (Ritchie and Lewis 2003 p. 79). The authors are aware of the disadvantages of purposive sampling regarding the missing statistical control of sample errors and uncertain generalization of the results. Nevertheless, a comprehensive requirements analysis demands the selection of reasonable experts, which is covered through a purposive sample. The criterions were covered within a sample of three experts, which were contacted by telephone and e-mail. All experts worked for a large international audit company. Further details of the experts and their background are presented in Table 1. Interview Approach The interviews were conducted as semi-structured interviews with an interview guideline. This approach was chosen by the following reasons: Since the authors have reasonable experience in the field of process audit as well as the experts, the authors expected an open conversation with the interviewees according to BURGESS and KVALE (Skinner 2012 p. 8). A semi-structured approach enables to change the order of questions and ask additional questions depending on the flow of the conversation. Further, a semistructured approach gives the experts the room to speak with more detail on the topic and enables them to introduce issues, which are relevant from their perspective (Oates 2005 p. 186ff.). A structured interview approach seemed to be inappropriate and far too restrictive for expert interviews. An unstructured interview approach would neglect the research questions, which set the frame for the investigations. Due to time and cost restrictions, the interviews took place on three different dates by telephone. The authors are aware that telephone interviews are constrained by words and voice (Genovese 2004). However, since the interviews focus upon the content, the consideration of body language and face expression are secondary. Since all experts are German, the interviews were conducted, recorded and transcribed in German. Afterwards, the answers were juxtaposed and standardized per question. The results are analyzed in the related sections. The applied interview guideline is also shown in Figure 2. Twentieth Americas Conference on Information Systems, Savannah, 2014 5

Accounting Information Systems Criterion Expert 1 Expert 2 Expert 3 Years of Experience 8 13 2,5 CISA exam Yes Yes Yes Industries Banking Manufacturing/Banking Banking Internal or External Internal/External External External Grade Senior Auditor Audit Manager Junior Auditor Interview Time 52:51 54:58 49:39 Table 1: Details of the Experts and Interviews Holistic and Integrated Specification of Process Audits The specification of process audits is derived from expert s interviews (Q1, descriptive). The answers were subsumed and placed in four different phases, which are related to the common standard ISA 315 (prescriptive). The legal framework (restrictive) has to be considered in all phases. The result of this work is shown in Figure 3. However, since there is no specific law for process audits, it is not pictured in this figure. In Phase 1 the auditor has to conduct a risk assessment regarding the specific risks of the audited entity and the industry. In some cases this phase has to be done prior to the order acceptance (Johnstone 2000). Sources for gaining a reasonable and appropriate understanding of the risks could be public media, such as professional journals, and industry reports as well as entity specific data, such as financial statements or financial position reports over time. The assessment of the risk concerning the probability of exposure and severity (i.e. expected loss) has to be conducted through the auditor. In Phase 2, the auditor should be aware of the entity, its environment and, most importantly, the entity s internal controls. Within the entity, this information is either written down explicitly in natural language, in textual operational processes, job descriptions etc. or tacitly available in the employees expertise (Nonaka 1991). However, in any case the auditor has to tap the expert s knowledge, which could be done by identifying and interviewing experts as well as demanding and recapping relevant documents. In this phase, the auditor is highly dependent on the client s cooperation regarding their willingness and capability to share knowledge. Entities could operate Knowledge Management Systems, such as Experts Databases, Knowledge Databases, enterprise wikis, IT management repositories or Document Management Systems, which support the activities (Alavi and Leidner 2001; Majchrzak et al. 2013). Further, the gained understanding should be appropriately visualized, i.e. the graphical representation has to consider auditor s scope. In the first instance, this contains the input and output of financial data as well as its processing and, most of all, the integrated controls within the process. This representation should be checked by the client, to validate the gained understanding. The results of Phase 2 are generally documented in a Risk Control Matrix (RCM). The RCM contains the material risks and compares them to the related integrated controls. Additionally, further information of the nature of controls is added such as frequency, preventive or detective control and automated or manual control. With respect to this insight, the auditor has to evaluate the adequacy and effectiveness of the controls in Phase 3. The evaluation of adequacy depends on the ability of the controls to reduce the addressed risk. Whereas the evaluation of adequacy depends on the collected information and the professional judgment of the auditor, the effectiveness of the control has to be proved by the Test of Controls. This can be conducted through interviews, observations, inspections or simulation of the control. Again, the client s cooperation is required to provide access to the processing systems, protocols or further evidences, which are produced by the operating systems. Finally, in Phase 4, the results of the audit have to be documented in an appropriate manner considering the size and complexity of the used methodology and results. The phases, the client s support and related systems are shown in Figure 3. 6 Twentieth Americas Conference on Information Systems, Savannah, 2014

Systematic Requirements Analysis and Design for Process Audit ISA 315 (Prescriptive) Expert s Interviews (Descriptive) Process Audit Auditor s Activities Client s Support Related Client s Systems Risk Assessment (ISA 315 A1 A16) Expert 1: Gain understanding of audited area Identification of risks Development of a method Expert 2: Identification of critical factors ( What can go wrong? ) Identification and assessment of risks (Documentation) Carve out specific Risks for Entity and Industry Evaluate Probability of Risk Financial Statements Financial Positions Accounting Information System Expert 3: Consideration of risks through scoping of audit area Evaluate Expected Loss of Risk Industry Reports Expert 1: Questioning of Experts Inquiry Expert Database Compilation of organizational rules Understanding of Entity s Environment and Internal Control (ISA 315 A17 A104) Identification of controls Expert 2: Compilation of organizational rules Identification of key controls (Documentation) Expert 3: Identification of key controls Identification of influence of key controls on control environment Demand Relevant Documentation Searching Relevant Documents IT Management Repository Knowledge Database Document Process Check Process Documentation Document Management System Expert 1: Evaluation of Controls (ISA 315 A105 A130) Evaluation of adequacy Test of Controls Conclusion Expert 2: Walk through process, evaluation of adequacy Identification of gaps Test of Controls (Documentation) Create RCM Test of Controls Support Test of Controls Front End System Accounting Information System Expert 3: Test of Controls Documentation (ISA 315 A131 A134) Expert 1,2 & 3: Documentation Document Results Reporting System Figure 3: The Phases of Process Audit according to ISA 315 and Expert s View Twentieth Americas Conference on Information Systems, Savannah, 2014 7

Accounting Information Systems Requirements Analysis of the Expert Perspective Relating to RQ1, essential objectives of the expert interviews were the questions Q2.1 and Q2.2. In the following analysis, the expert s answers are categorized either in overall requirements or into the phases, shown in the prior section. Overall Requirements Expert 1 and 2 have mentioned that both legal framework and professional standards are sufficient and indispensable conditions for the conduction of a process audit. Therefore, the consideration of legal requirements is defined as requirement R1. In this context, all experts appointed the application of an appropriate method as an important requirement (R2). Further, Expert 1 and 3 pointed out that the experience in process audits and the ability to adapt this experience from similar audits to the current task to be considered. In this sense, R3 is defined as the consideration of the auditor s experience. From a broader perspective on process audit, all experts named the consideration of budget constraints (i.e. time and cost) as important requirements (R4). Risk Assessment For all experts, the conduction of a risk assessment as a first phase in process audit is necessary. In depth, risk assessment focuses on the identification of essential risks and the evaluation of the probability and severity of the risks. Expert 2 stated, that this assessment has to be done prior the scope of the process is broadened. Therefore, the requirement R5 is derived as the essential risks are identified and evaluated. Understanding of Entity s environment and internal control All experts describe the cooperation with the audited client as essential for the understanding of the entity s environment. Expert 1 believes that the client has to hold the organizational regulation as well as the descriptions of the processes and systems ready and up-to-date. For Expert 2 is the identification of internal key controls indispensable. For all experts the conduction of interviews to collect the necessary information is important. Thus, the client s domain experts must have the knowledge of the process in scope as well as the willingness to share this knowledge and, finally, the willingness to answer the auditor s question in a truthful and in appropriate manner (Expert 1, 2, 3). The derived requirements are: The client has up-to-date organizational regulations (R6). The client s domain experts are identified (R7.1) and they share their knowledge in a truthful and in appropriate manner (R7.2). Furthermore, all experts stated that the gained understanding has to be documented in graphical and textual representation (R8). Evaluation of Controls Since the evaluation of Controls (i.e. Test of Controls) is performed by inquiries and observations, the previously mentioned requirements R6-R8 are valid for this phase. Expert 1 and Expert 3 stated that the effectiveness of controls could be tested by the re-performance of the controls. Thus, the auditor needs the access to the financial data of the operating AIS (R9). Furthermore, the auditor needs the appropriate methods to evaluate the controls within the AIS (R10). Documentation For Expert 2 documentation is not a final activity in the process audit. Instead documentation is continuous and is developed attendant to the whole process. However, Expert 1 and Expert 2 stated, that the documentation has to be proper and comprehensible for third parties, which is the next requirement: transparent and comprehensible for third parties (R11). Expert 3 stated that documentation of previous years has to be reusable, which constitutes the 12 th requirement or reusability in the following years (R12). 8 Twentieth Americas Conference on Information Systems, Savannah, 2014

Systematic Requirements Analysis and Design for Process Audit Phase No. Requirement Current CAATTs Support Potential CAATTs Support Overall R1 R2 R3 consideration of legal requirements application of an appropriate method consideration of auditor s experience R4 consideration of budget constraints Risk Assessment Understanding of Entity s environment and internal control Evaluation of Controls Documentation R5 R6 R7.1 R7.2 R8 R9 R10 essential risks are identified and evaluated client has up-to-date organizational regulations client s domain experts are identified client s domain experts share their knowledge in a truly and appropriate manner understanding has to be documented in graphical and textual representation access on financial data appropriate methods to evaluate the controls within the AIS transparent and comprehensible for R11 third parties R12 reusability in the following years Legend: = low, = medium, = high Table 2: Requirements of Process Audit and Support through CAATTs Discussion of CAATTs Support With respect to RQ2, for all requirements support through CAATTs and limitations are discussed. Expert s thoughts about possible developments of innovative CAATTs are based on the questions Q3.1 and Q3.2. The results are summarized in Table 2. For the time being, the experts do not know any CAATTs, which support R1. The development of CAATTs is difficult since legal framework is manifold and decentralized. Further, there is no specific law or standard for process audits. For the development of CAATTs in this field, the authors suggest its exploration in future work. Since the auditor s experience (R3) is tacit and a result of practical work the computer-assistance is limited on E-Learning and knowledge databases. Therefore, the development of reasonable CAATTs is strongly related to the field of knowledge transfer, which is the objective of versatile research work. R4 arises from the project nature of the audit and is not directly related to the tasks of process audit. Hence, this requirement is supported by pre-existing project management tools. Twentieth Americas Conference on Information Systems, Savannah, 2014 9

Accounting Information Systems Surprisingly, all experts stated that in the field of financial audits, CAATTs for carving out and evaluating risks (R5), such as tools for prioritizing areas of higher risks by observing rating changes or impairment tests (Expert 2) are seldom used. Further work has to consider existing techniques of risk assessment. For R6 and R7, the experts do not know any CAATTs. Assuming that the client operates organizational regulation and descriptions of processes and systems in knowledge management systems, Expert 3 sees high potential for the development of CAATTs if decentralized information can be combined for the auditor s purposes. An example of this is the automatic synchronization of process models and job descriptions. To support R7.1, an expert database with an integrated recommender system is within the realm of possibility. R7.2 is strongly related to the auditor s social skills. Therefore, developing appropriate CAATTs seems impossible, for the time being. Existing CAATTs enable the analysis of processing within the AIS (Braun and Davis 2003; Hall 2010), either by analyzing the outcome of the AIS (R9) or through a direct examination of the application logic (R10). Due to growing data volume, the experts assume that the need for development of CAATTs will increase in this phase. Furthermore, an increased integration of CAATTs in the client s control environment should be strived for. R2, R8, R11 and R12 are subsumed since they are related with methods and documentation. R2 is currently supported by electronic guidelines and checklists. However, these techniques are characterized by an inflexible structure and low adaptability. R8, R11 and R12 are currently supported by documentation software and MS Office tools, which are characterized by heterogeneity and media discontinuity (Expert 3). Expert 1 proposed an automated documentation system, which visualizes the whole control environment of an audited area and produce a report, which contains all the audited controls along the process. Further, Expert 1 and 3 proposed the establishment of a standardized modeling language as well as modules with patterns of controls and sub-processes, i.e. a customizable audit tool that could increase traceability and reusability of the results. Conclusion and Outlook Innovative computer-assistance in audit is of increasing importance. Through interviews of three IT auditors and the consideration of the relevant standard ISA 315, the essential phases of process audits were carved out. Throughout these phases mandatory and other important requirements for successful process audits were examined and discussed in this paper. Further, it was discussed to what extent current tools and techniques cover these requirements and where gaps could be closed through computerassisted support. This sets the baseline of development of innovative CAATTs, which attempt to increase efficiency regarding high quality results (effectiveness). Limitations were shown in the development for legal framework and social related requirements. According to the principles of Design Science, in this paper areas were carved out, which demand the development of appropriate IT artifacts for audit purposes. In this paper, the baseline was set for researchers from different IS disciplines, such as modeling or software engineering, to investigate in these areas. Particularly, in the fields of risk assessment, client s cooperation, modeling support and production of comprehensible documentation is high potential for innovative CAATTs. In future work, more auditors from different industries will be consulted and single requirements will be soundly explored. Finally, the complete alignment between IT and audit should be strived for through the implementation of continuous audit with the intent of a permanent monitoring of the control environment while processing financial data. Particularly, the analysis of financial data should be examined with the respect to the Big Data Computing paradigm and its possibilities, such as Data Mining or Neural Networks. REFERENCES AICPA. 1972. Responsibilities and Functions of the Independent Auditor, United States of America: SAS No. 1, section 110; SAS No. 78; SAS No. 82, pp. 1 2. AICPA. 2001. Effect of information technology on the auditor s consideration of internal control in a financial statement audit, United States of America, pp. 31, 32. 10 Twentieth Americas Conference on Information Systems, Savannah, 2014

Systematic Requirements Analysis and Design for Process Audit Alavi, M., and Leidner, D. 2001. Review: Knowledge Management and Knowledge Management Systems: Conceptual Foundations and Research Issues, MIS Quarterly (25:1)Management Information Systems Research Center, University of Minnesota, pp. 107 136. Atkinson, R. 1999. Project management: cost, time and quality, two best guesses and a phenomenon, its time to accept other success criteria, International Journal of Project Management (17:6), pp. 337 342. Braun, R. L., and Davis, H. E. 2003. Computer-assisted audit tools and techniques: Analysis and perspectives, Managerial Auditing Journal (18:9)MCB UP Ltd, pp. 725 731. Buchanan, S., and Gibb, F. 1998. The information audit: an integrated strategic approach, International journal of information management (18:1)Elsevier, pp. 29 47. Bushman, R. M., and Smith, A. J. 2001. Financial accounting information and corporate governance, Journal of accounting and Economics (32:1)Elsevier, pp. 237 333. Carlin, A., and Gallegos, F. 2007. IT Audit: A Critical Business Process, Computer (40:7), pp. 87 89. Cerullo, M. V., and Cerullo, M. J. 2003. Impact of SAS no. 94 on Computer Audit Techniques, Information Systems Control Journal (1)INFORMATION SYSTEMS AUDIT AND CONTROL ASSOCIATION, pp. 53 58. Coderre, D., and Police, R. C. M. 2005. Global Technology Audit Guide: Continuous Auditing Implications for Assurance, Monitoring, and Risk Assessment, The Institute of Internal Auditors. Curtis, B., Kellner, M. I., and Over, J. 1992. Process modeling, Communications of the ACM (35:9), pp. 75 90. European Commission. 2013. Commissioner Michel Barnier welcomes provisional agreement in trilogue on the reform of the audit sector, MEMO/13/1171 Brussels. Gelinas, U., Dull, R., and Wheeler, P. 2011. Accounting information systems, Cengage Learning, p. 67. Genovese, B. J. 2004. Thinking inside the box: The art of telephone interviewing, Field Methods (16:2)Sage Publications, pp. 215 226. Goguen, J. A., and Linde, C. 1993. Techniques for requirements elicitation, in [1993] Proceedings of the IEEE International Symposium on Requirements Engineering,, pp. 152 164. Hall, J. 2010. Information Technology Auditing, (3 ed.) Mason, OH: South-Western Cengage Learning, pp. 314ff., 389ff. Hammer, M. 2007. The process audit, Harvard business review (85), p. 111. Hevner, A. R., March, S. T., Park, J., and Ram, S. 2004. Design science in information systems research, MIS Quarterly (28:1), pp. 75 105. Hickey, A. M., and Davis, A. M. 2003. Requirements elicitation and elicitation technique selection: model for two knowledge-intensive software development processes, in 36th Annual Hawaii International Conference on System Sciences, 2003. Proceedings of the,, p. 10 pp. Twentieth Americas Conference on Information Systems, Savannah, 2014 11

Accounting Information Systems IFAC. 2013a. ISA 200, in Handbook of International Quality Control, Auditing, Review, Other Assurance, and Related Services Pronouncements, (1st ed.) New York: International Federation of Accountants, p. 74. IFAC. 2013b. ISA 315, in Handbook of International Quality Control, Auditing, Review, Other Assurance, and Related Services Pronouncements, (1st ed.) New York: International Federation of Accountants, pp. 267 320. Information Systems Audit and Control Association. 2008. IS Auditing Guideline: G3 Use of Computer- Assisted Audit Techniques,. Janvrin, D., Lowe, D. J., and Bierstaker, J. 2008. Auditor acceptance of computer-assisted audit techniques, Iowa State University, Arizona State University and Villanova University (4). Johnstone, K. M. 2000. Client-Acceptance Decisions: Simultaneous Effects of Client Business Risk, Audit Risk, Auditor Business Risk, and Risk Adaptation, AUDITING: A Journal of Practice & Theory (19:1)American Accounting Association, pp. 1 25. Mahzan, N., and Lymer, A. 2008. Adoption of computer assisted audit tools and techniques (CAATTs) by internal auditors: current issues in the UK, in BAA Annual Conference,, pp. 1 46. Majchrzak, A., Wagner, C., and Yates, D. 2013. The Impact of Shaping on Knowledge Reuse for Organizational Improvement with Wikis, Management Information Systems Quarterly, (Vol. 37), pp. 455 469. McDaniel, L. S. 1990. The Effects of Time Pressure and Audit Program Structure on Audit Performance, Journal of Accounting Research (28:2)Wiley on behalf of Accounting Research Center, Booth School of Business, University of Chicago, pp. 267 285 CR Copyright 1990 Accounting Res. Nonaka, I. 1991. The knowledge-creating company, Harvard business review (69:6), pp. 96 104. Oates, B. J. 2005. Researching information systems and computing, Sage. Ritchie, J., and Lewis, J. 2003. Qualitative research practice: A guide for social science students and researchers, Sage. Ross, D. T., and Schoman, K. E. 1977. Structured Analysis for Requirements Definition, IEEE Transactions on Software Engineering (SE-3:1), pp. 6 15. Sabahat, N., Iqbal, F., Azam, F., and Javed, M. Y. 2010. An iterative approach for global requirements elicitation: A case study analysis, in 2010 International Conference on Electronics and Information Engineering, (Vol. 1), August, pp. V1 361 V1 366. Samson, T. F. 1973. COMPUTER AUDITING, EDPACS (1:3)Taylor & Francis, pp. 17 18. Sarbanes-Oxley Act. 2002. Public Law No. 107-204, Washington, DC: Government Printing Office (107). Sayana, S. A. 2003. Using CAATs to support IS audit, Information systems control journal (1)INFORMATION SYSTEMS AUDIT AND CONTROL ASSOCIATION, pp. 21 23. 12 Twentieth Americas Conference on Information Systems, Savannah, 2014

Systematic Requirements Analysis and Design for Process Audit Schultz, M., Müller-Wickop, N., and Nüttgens, M. 2012. Key Information Requirements for Process Audits an Expert Perspective, in Proceedings of the 5th International Workshop on Enterprise Modelling and Information Systems Architectures, Vienna, pp. 137 150. Skinner, J. 2012. The Interview: An Ethnographic Approach, (Vol. 49) London, UK: Bloomsbury Publishing, p. 8. Taylor, E. Z., and Dzuranin, A. C. 2010. Interactive financial reporting: an introduction to extensible business reporting language (XBRL), Issues in Accounting Education (25:1), pp. 71 83. Tiwari, S., Rathore, S. S., and Gupta, A. 2012. Selecting requirement elicitation techniques for software projects, in 2012 CSI Sixth International Conference on Software Engineering (CONSEG),, September, pp. 1 10. Vasarhelyi, M. A., Chan, D. Y., and Krahel, J. P. 2012. Consequences of XBRL standardization on financial statement data, Journal of Information Systems (26:1)American Accounting Assocation, pp. 155 167. Vasarhelyi, M. A., and Halper, F. B. 1991. The continuous audit of online systems, Auditing: A Journal of Practice and Theory (10:1), pp. 110 125. Zabihollah Rezaee, Rick Elam, A. S. 2001. Continuous auditing: the audit of the future, Managerial Auditing Journal (16:3), pp. 150 158. Twentieth Americas Conference on Information Systems, Savannah, 2014 13