Establishing Production Capability (PC) to use Automated Data Analysis Tools and Resources To increase the effectiveness of the Citizens Internal Audit effort. Prepared By: I. Steve Wolkomir CPA Citizens Director of Internal Audit File: Audit Planning Establishing PC Data Analysis Tools 1
What is the purpose of using Data Analysis Tools? To proactively identify control issues ideally before they impact Citizens. To search for areas of possible cash recovery. Ask: What areas within Citizens might we have potential for cash recovery? To use technology to improve business processes. Manual process analysis takes too long. Maximizes/leverages IA s time. What types of Data Analysis Tools are available and what are their characteristics? ACL (IDEA is similar): Can validate integrity and accuracy of transactions at the transaction data level (i.e., raw data), versus using and relying upon older data queries; thus providing business assurance. Can access entire population of data, with no limitation, and is platform independent. Is a parameter based system; must know what you want to achieve. Has rather steep learning curve to learn basic and then learn to script or write applications. As a result, once begun, it must be used often. Model is to have dedicated staff to use and apply. Adapts well to Continuous Audit Assurance; i.e., once a test set or script is developed, it can be run multiple times at little additional cost. 2
Tools availability/characteristics (Cont d)? WizRule: A Data Mining auto-rule generator tool. Nearly the opposite approach to a parameter based tool as Data Mining is defined to be: The nontrivial extraction of implicit, previously unknown, and potentially useful information from data. It uses machine learning, statistical (e.g. the tool ActiveData for Excel; a possible useful tool for IA) and visualization techniques (e.g. the tool BI3) to discover and present knowledge in a form easily comprehensible to humans. This tool finds all rules in a database and tries to break them; i.e., to find exceptions to the rule(s)/norm. Exceptions then can be further analyzed and investigated. It assumes, as does those knowledgeable in data bases, that there are a great number of errors that occur in data bases; whether unintentional or intentional or through malfunction. Unlike ACL, this tool is simple to use. Like ACL, this tool adapts very well to being used continuously. 3
Tools availability/characteristics (Cont d)? All of these tools have a few things in common: They all facilitate finding uncommon exceptions. They all augment and support audit, review and investigative efforts. They all support a much more continuous look at a set of data or control points, versus the traditional approach of looking at data/control points at a point in time; with an historic backward look. They all are Heuristic in nature; An art and science of discovery. A way of directing our attention much more fruitfully. A way of reviewing data base populations down to the transaction level; a practical impossibility otherwise. Acts like divining rods that don t necessarily provide an exact answer, but rather point us in the right general direction. Note: ACL, being more of a parameter tool, will be more likely to produce more specific answers than the more pure Data Mining tools. 4
IA s Action Plan is as follows: Committed to establishing PC to use Data Analysis tools. With regard to ACL, IA s Plans are; Typical traditional approach would be to; Buy the software $2k to purchase and $420 per year to warrant and obtain customer support. Train on the software» Attend vendor seminars ($1,300, $1,350, and $1,650)» Buy books.(see below for test sets)» Talk to users. Apply using Trial and Error; very time consuming with IA having no time for this. Run Reports, with no certainty of success. IA s approach will be a paradigm shift; Run Reports» Will, at no cost, obtain ACL Beta license for 60 days.» Will purchase two books for $200 per from a vendor who specializes in writing test sets/scripts. The $400 will include over 100 test sets in both A/P and Revenues.» Depending on the approved Audit Plan, IA will run selected reports either ad hoc or within the scope of an audit.» IA will then report on the results of this testing. This up-front method effectively eliminates the Trial and Error period shown above. After some successes, IA will purchase the software, and through training or other means, will work toward developing test sets tailored to Citizens unique environment. This may include;» Purchasing tailored test sets from the above mentioned vendor who bills his work at $125/hr. This vendor would be given access to our data base in a controlled manner, extract data, and analyze that data. Again, the end result is what IA is focused upon. This approach lets the computer do the data analysis; with IA concentrating on results. Advantages include; These pre-developed test sets were developed to get fast results. The test sets include;» Audit objectives, Audit Program Steps, and Reports that can be s elected to accomplish specific Audit Objectives. Minimizes risk and investment. Can be run continuously with little IA time. With regard to WizRule; IA plans on buying this tool and using it as soon as possible. With regard to ActiveData for Excel, IA plans on looking into the purchase/use of this tool. 5
Any questions or comments? 6