Neural networks (NNs) are becoming more commonplace

Transcription

1 Copyright 2006 ISACA. All rights reserved. Using Neural Network Software as a Forensic Accounting Tool By Michael J. Cerullo, Ph.D., CPA, CITP, CFE, and M. Virginia Cerullo, Ph.D., CPA, CIA, CFE Neural networks (NNs) are becoming more commonplace in real-world business applications and in some cases, such as fraud detection, they have already become the method of choice. 1 NNs are also being employed in risk assessment and internal controls, as well as to visualize complex databases for marketing segmentation. The boom in applications covers a wide range of business interests from finance management, to forecasting, to production. The combination of statistical, neural and fuzzy methods now enables direct quantitative studies to be carried out without the need for rocket-science expertise. However, the tremendous potential of NNs in business and accounting has yet to be realized. 2 Neural networks are becoming increasingly adaptable; thus, their use is expected to become more common and widespread in time. This article illustrates how NN software packages can be utilized by firms to predict the occurrence of financial statement fraud. Specifically, the following topics will be examined: The nature of NNs Reasons to use NNs The structure of a neural network Detecting financial reporting fraud with an NN NN technology equips auditors with expertise that previously could only have been attained with years of training and experience. 3 Auditors often find that financial statement fraud is very difficult to detect because it is subject to concealment or falsification of accounting records and supporting documents. Thus, a possible strategy for auditors is to proactively assess the likelihood of such fraud. This ability to accurately assess the risk of fraud is critical to the initial assessment of risk of material misstatement during the initial planning phase of the audit. The risk assessment will affect the design of subsequent audit testing techniques. 4 The Nature of Neural Networks As has been mentioned numerous times in the periodic literature, an artificial NN processes data similarly to a human brain. Specifically, an NN is a type of nonlinear statistical analysis program that utilizes a learning process (trial and error) to repeatedly analyze a collection of historical data sets to recognize patterns in that data and to automatically produce a model for that data. It processes data the way the brain processes data in a multiple parallel processing mode. This multiple processing capability enables NNs to execute operations much faster than traditional methods, which process data serially, one operation at a time. NNs solve problems by recognizing patterns in data that may be too subtle or complex for humans or other types of computer methods to discern. Basically, an NN creates a mathematical model from a historical database of examples of input and output values. After learning the relationship between the variables, the network has been trained, and a mathematical model is constructed that recognizes patterns in the sample data, such as correlations between seemingly unrelated data. The resulting model, when used with new input data, provides projections of future outputs. For example, by collecting historical data of commercial loans made to organizations, a bank can determine which organizations have defaulted on repaying the loans. A model can be built that contains the relationship, if any, between a firm s selected financial ratios and the outcome of the loan. After the model is synthesized, it can be used to predict if a new commercial loan applicant is likely to default on repayment. Reasons to Use Neural Networks As indicated by figure 1, an NN is suited for simple to complex, and structured to unstructured, problems. Thus, an NN can solve a much broader range of problems than, for example, an expert system, including problems that are almost completely random in nature. Specific fraud applications suitable for development by using NNs can be categorized into prediction, classification and pattern recognition problems. In recent years, NN technology has been used in a few firms to develop models for various forms of fraud detection. A recent study provides confirmation that NN techniques are an effective method for developing a fraud classification model to assess the risk of errors and irregularities in financial statements. 5 The variables used in developing a particular fraud detection model may differ from one company/industry to another or according to the targeted type of fraud. NNs, with their remarkable ability to derive meaning from imprecise data, can be used to detect trends that are too complex to be noticed by humans or other computer programs, including those depicted in figure 1. A trained NN can be thought of as an expert in the category of information it has been given to analyze. This expert can then be used to provide projections given new situations of interest and answer what if questions. Other advantages include: 6 Adaptive learning An NN can learn how to do tasks based on the data given for training or initial experience. Self-organization An NN can create its own representation of the data set it receives during training time. Real-time operation An NN can rapidly go through millions or billions of simulations to train the data set in parallel. Fault tolerance via redundant information coding A partial destruction of a network leads to the corresponding degradation of performance. However, some network capabilities may be retained even with major network damage. Developing and implementing an NN application is often much easier and faster than developing an accounting expert system application since most commercially available NN software packages automatically select the network structure. 1

2 Figure 1 Practicality of Applying an NN to Solve a Large Class of Problems Figure 2 Schematic of a Simple Neural Network Structure Complex Problem Complexity Expert Systems Neural Networks Input 1 i W h Z O Output Simple Input 2 Conventional Software Certain Statistics Moderately Uncertain Very Uncertain Level of Uncertainty Completely Random Normally, the entire developmental process from start to implementation can be completed in a few weeks. Neural Network Structure Figure 2 presents a simple NN structure, which is comprised of multiple inputs and a single output. The basic building block in an NN is the neuron, depicted by the circles (also called nodes). As illustrated, every NN has an input layer (i), a hidden layer (h) and an output layer (o). Each artificial neuron in a layer receives its input from the output of the previous layer nodes or from the network inputs. The connections between nodes are associated to adjustable weights (W, Z) that are adjusted as the network is trained. Each neuron in the network processes the input data, with the resulting values steadily seeping through the network layer by layer, until a result is generated in the output layer. Using Neural Networks to Detect Financial Statement Fraud: A Case Study The stages to develop an NN to predict fraud (see figure 3) are: Formulate the problem. As shown in figure 3, the first step is to select a problem that is suitable for an artificial NN. As mentioned, suitable business applications fall into three categories: 1. Classification problems Fraud detection, loan approval, loan default and credit card applications fall into this category. 2. Time series applications Financial forecasting, stock market prediction, bankruptcy prediction, bad debts estimation, sales and expense forecasting are examples of business applications in this category. 3. Data mining applications Many marketing applications fall into this category, such as looking for patterns in customer databases, targeting customers, estimating responses and analyzing demographics. This article will provide a case study of using an NN for the classification category involving fraud committed by an organization s top management. Such frauds are more commonly called cooking the books or financial statement fraud. This type of fraud is commonly committed by overstating revenues and understating liabilities or expenses. According to the Association of Certified Fraud Input 3 Examiners (ACFE s) 2004 Report to the Nation on Occupational Fraud and Abuse, the average financial statement fraud reported by survey respondents is over US $1 million. Financial statement frauds, such as the WorldCom and Enron frauds, can overstate income by billions of US dollars. A variety of data analysis techniques can be employed to detect financial statement fraud, including Benford s law, horizontal and vertical analyses, ratio and trend analyses, and NN software. NN software is an often overlooked but very powerful and easy-to-use tool in detecting management fraud. Figure 3 Detecting Fraud With a Neural Network Formulate problem. Create database of examples. Construct model by training database. Evaluate model s performance. Implement model, if valid. 2

3 Assume that Dale Rogers, CPA, CFE, partner in a regional Denver, Colorado, USA, CPA financial services firm is using the ModelQuest TM Enterprise NN package (hereafter referred to as ModelQuest) to automatically create a financial statement fraud prediction model for clients. 7 Other NN packages include Brainmaker (from California Scientific Software) and NeuroSolutions (from Neuro Dimension Inc.). Assume that Rogers has acquired a good working knowledge of the package and plans to use it in her audit engagements. She plans to use the NN model during initial audit planning and the preliminary assessment/ analytical review phase to aid in predicting if financial reporting fraud has been perpetrated in client companies. Create the database. Now that the problem is defined, her next step is to create a database of historical numerical examples input into the NN software to generate a fraud prediction model. The database must contain independent input variables and corresponding dependent output variables. Creating the database is the most important step in the NN developmental process. Unrepresentative sample data or too few data observations will result in a model that poorly estimates or predicts future values. Assume Rogers next goes to the US Securities and Exchange Commission s (SEC s) web site to conduct an online search of potential financial ratios indicating financial reporting fraud. Based on her readings of several research studies on the usefulness of financial ratios to predict fraud, she selects 26 ratios (i.e., the independent variables, X) that have been employed by companies to predict possible fraudulent financial reporting (i.e., the dependent variable, Y). The ratios selected are given in figure 4. Based on discussions with expert auditors in her firm, Rogers selects the nine ratios (in bold letters in figure 4) to use in her fraud prediction model. Next, Rogers does an Internet search of national newspapers, ACFE, the American Institute of Certified Public Accountants (AICPA), the US Department of Justice and the Stanford Research Institute, and finds 15 firms that have reported financial reporting fraud and an additional 15 companies that have not reported such fraud. Her next step is to conduct an online search of the SEC s EDGAR database. From these data, she creates a database of the previously mentioned nine ratios for the 30 firms for the year before any of the frauds were reported. The database of financial ratios is presented in figure 5. The nine ratios are known as the X, or independent, variables, and the default is the Y, or dependent, variable. Since symbolic values cannot be processed by NN software, they are converted to the numerical values 0 or 1. To illustrate the concept, firm A shows a 1 for the default (dependent) variable. This means that financial statement fraud was uncovered at that firm; likewise, a value of 0 for firm B means that, as of the reporting date, no financial statement fraud was reported at that firm. Construct model. Next, the table of input and output values is keyed into the ModelQuest NN package to construct a network depicting the relationships of the inputs to the output. Rogers will have the software automatically split the table into a training subset of 22 observations and a test subset of eight observations. The NN software will use the training file to train the network into a mathematical model Figure 4 Potential Ratios Used to Predict Financial Statement Fraud Earnings/assets Equity/debt Current ratio...current assets/current liabilities Acld-test ratio...quick assets/current liabilities Receivables turnover...net sales on account/average receivables Number of day s sales in...receivables end of receivables year/average daily sales Inventory turnover...cost goods sold/average inventory Number of days sales in...inventory end of year/average inventory daily CGS Net fixed assets/long term liabilities Debt to equity...total debt/stockholders equity Net sales/average total assets Rate earned on total assets...income + interest expense/average total assets Rate earned on stockholders...income/total stockholders equity equity Cash flow/total debt Cash flow/long-term debt After-tax profit/total assets Total liabilities/total assets Net working capital/total assets Net working capital/sales revenue Return on assets...income/average total assets Credit rating...provided by a credit rating agency Bond rating...1 = low, 10 = high Sales ratio...sales/assets Earnings before interest and taxes (EBIT)/assets Profit margin or efficiency ratio Income/sales Equity/assets that recognizes patterns between the independent variables (inputs) and the dependent variable (i.e., fraud or no fraud). The test data determine how well the synthesized network model is able to generalize on new, unseen data. Evaluate performance. The next step is to evaluate the performance of the network. Rogers evaluates the model constructed by comparing the output summary statistics for the test data subset with the training data subset. Figure 6 shows hypothetical summary output statistics for the database given in figure 5. 8 Generally speaking, the average absolute error (AAE) and maximum absolute error (MAE) are two good indicators to evaluate when determining if the model constructed is acceptable. 9 The AAE is the sum of the errors, ignoring the plus or minus signs and averaging the result. The MAE is the maximum amount of error between the predicted and actual values. In this example, the AAE (.23) is low compared to the output maximum (1.0) and minimum (0.0), which indicates that, at first glance, the model appears to be acceptable. The MAE indicates the worst the model did on the subset of testing or evaluation data. In this illustration, sometimes the model will generate completely erroneous results. 3

4 Figure 5 Hypothetical Database Used to Predict the Presence or Absence of Financial Statement Fraud Name of Firm EquityAsset EarnAsset EquDebt CurrRat QuickRat BondRat ROA EBITAsset ARTurn Default A B C D E F G H I J K L M N O P Q R S T U V W X Y Z A B C D It is important to also compare the output minimum, maximum, mean and standard deviation. If these numbers are close to one another between the testing and training data sets, the synthesized model is usually acceptable. On the other hand, if these values are quite different, the number of observations should be increased and a new model should be trained. Also, the predicted squared error (PSE) should be compared to the average squared error (ASE). 10 The two values should be relatively close to one another. In this case, the PSE is 0.10 and the ASE is Finally, training results were evaluated by computing the R 2 value, the coefficient of determination. An R 2 approaching 1 indicates a good model fit and an R 2 near 0 indicates a poor fit. Predicted output in the testing data set formulated by the NN matched the training output with a correlation of Figure 6 shows that if the evaluation of the output summary statistics indicates that an unacceptable model was developed, Rogers must try new variables, increase the sample size, retrain a new model and evaluate the output statistics. If the results still appear to be unacceptable, an NN probably cannot be constructed for this problem situation. Implement model. Assume that Rogers gets acceptable results and that all nine input values are significant. The model can now be implemented for its intended purpose of predicting fraudulent financial reporting. On an audit engagement, during the initial audit planning and preliminary assessment/analytical review phase, Rogers would compute the nine financial ratios for the previous year and input the nine values into ModelQuest using the package s query function. Assume that the model immediately computes a 1 output value from the inputs, which indicates that the firm Figure 6 Hypothetical Output Summary Statistics for the Fraud Prediction Model RESULTS TEST DATA TRAINING DATA Number of observations 8 22 Minimum 0 0 Maximum Output mean Output standard deviation.5.45 Maximum absolute error Average absolute error Predicted squared error -.10 Average squared error R has possibly engaged in fraudulent financial reporting and this potential red flag must be further investigated. Summary and Conclusions NNs are frequently ignored by internal and external auditors as a major data analysis tool that can be effectively employed to predict the occurrence of fraudulent financial reporting. This article presented a case study of how a hypothetical regional public accounting firm employed an NN software package in client engagements to aid in predicting if financial statement reporting fraud is likely occurring. NNs process large amounts of data to solve problems by recognizing patterns, trends and relationships that may be too subtle or complex for humans or other types of computer methods, such as statistical models, to discern. 4

5 Basically, an NN computer software model is constructed and trained from a database of historical examples of input and output variables. During model training, an NN learns the patterns and correlations from a sample of input and output data representing actual fraud occurrences and nonfraud occurrences, respectively. The creation of the database is the most important step in the NN developmental process. The input and output values are keyed into the NN package to construct a network that can then be used by an accountant to make a decision about the possible occurrence of fraud. The ModelQuest NN package was used to construct the mathematical model for the case study discussed because the package can be mastered in a few hours and requires only a rudimentary understanding of statistical concepts. Other NN packages are similar to ModelQuest. The model developed by the ModelQuest software can be used in future engagements to predict whether financial reporting fraud, the dependent variable, has been perpetrated in client companies. The key to managing this type of fraud and reducing its large potential losses is to employ the most effective tools available to fight it. Many firms are utilizing new information technologies to address old problems, such as fraud. Auditors have a tremendous opportunity to learn about and employ NNs to ensure that their firms or clients remain competitive. Endnotes In the authors experience, of all the data analysis methods used to predict or detect fraud, including general audit software, statistical software, ratio analysis, spreadsheets, relational database packages, financial statement analysis software and Benford s law, NN software is the method least often employed. For example, in a study by the authors, about 7 percent of internal auditors responding to a survey used NNs vs. more than 45 percent who used general audit software. (See Cerullo, Michael J.; M. Virginia Cerullo; Tracy Hardin, Computer Techniques Used to Audit the Purchasing Function, Internal Auditing, March/April 1999, p. 24.) Also, even more shocking, a recent article revealed that, of 400 auditors responding to the 10 th IIA annual software survey, none used NN software. (See Jackson, Russell A.; Get the Most Out of Software Tools, Internal Auditor, August 2004, p , 44 and 47.) Finally, a 2004 study of auditors use of technology published on the IIA web site ( revealed that, of 14 technologies surveyed, NNs ranked dead last. 3 One of the authors has successfully used an NN package for the past 12 years in a graduate IS auditing course. Most students learn how to physically utilize the software within a few hours; in about another two hours, they acquire a good working knowledge of the statistics employed to evaluate models. They then complete a case study on the likelihood that commercial loan applicants will default on their loans. 4 Lin, Jerry W.; Mark I. Hwang; Jack D. Becker; A Fuzzy Neural Network for Assessing the Risk of Fraudulent Financial Reporting, Managerial Auditing Journal, August 2003, p Green, Brian Patrick; Jae Hwa Choi; Assessing the Risk of Management Fraud Through Neural Network Technology, Auditing, Spring 1997, p Adapted from journal/vol1/cs11/article1.html 7 This package is available from AbTech Corporation, 1575 State Farm Blvd., Suites 1&2, Charlottesville, VA 22911, Basically, all NN software packages provide similar statistics. Most of the statistics concern evaluating various measures related to the errors or residuals. Errors or residuals are the difference between the actual output values and the predicted output values. Ideally, the closer that the sum of the errors is to 0, the better the model. 9 Adapted from ModelQuest Expert: User s Manual, Charlottesville, VA, Abtech Corporation, 1997, p. 8-2 and These individual differences are squared, summed and averaged. The result is the ASE. 11 The PSE is computed from the training data subset and is an approximation of the expected network squared error (ASE) for data not in the training subset. That is, the model predicted that the ASE on new data should be.10; in fact, the ASE on the test data (i.e., new data) was When training results do not indicate a high correlation, one option is to try other network architectures available with the ModelQuest software. Michael J. Cerullo, Ph.D., CPA, CITP, CFE is a professor of accounting at Southwest Missouri State University, Springfield, Missouri, USA. He specializes in accounting information systems, information systems auditing and fraud examination. He has published approximately 150 articles in professional and academic journals. He received his doctorate from Louisiana State University. M. Virginia Cerullo, Ph.D., CPA, CIA, CFE is a professor of accounting at Southwest Missouri State University, Springfield, Missouri, USA. She specializes in internal auditing and is the coordinator of the Institute of Internal Auditor s Endorsed Internal Audit Program at SMSU. She received her doctorate from Louisiana State University and has published numerous articles in professional and academic journals. Information Systems Control Journal is published by ISACA. Membership in the association, a voluntary organization serving IT governance professionals, entitles one to receive an annual subscription to the Information Systems Control Journal. Opinions expressed in the Information Systems Control Journal represent the views of the authors and advertisers. They may differ from policies and official statements of ISACA and/or the IT Governance Institute and their committees, and from opinions endorsed by authors employers, or the editors of this Journal. Information Systems Control Journal does not attest to the originality of authors' content. Copyright 2006 by ISACA. All rights reserved. Instructors are permitted to photocopy isolated articles for noncommercial classroom use without fee. For other copying, reprint or republication, permission must be obtained in writing from the association. Where necessary, permission is granted by the copyright owners for those registered with the Copyright Clearance Center (CCC), 27 Congress St., Salem, Mass , to photocopy articles owned by ISACA, for a flat fee of US $2.50 per article plus 25 per page. Send payment to the CCC stating the ISSN ( ), date, volume, and first and last page number of each article. Copying for other than personal use or internal reference, or of articles or columns not owned by the association without express permission of the association or the copyright owner is expressly prohibited. 5