Our Data Analytics Journey, Methodology, and More September 15, 2015
Objectives High-level Objectives: Discuss Audit Data Analytics History Industry Personal History TIAA-CREF History Define our data analytics integration process Discuss how to consume data analytics and mitigate consumption risk Read / interpret results and follow-up procedures and questions Discuss results with the business Best practices and common pitfalls Discuss and display recent analytic successes 2
Industry History Industry History Late 1980s generalized auditing software companies form ACL, 1987 Caseware, 1988 Charles Carslaw, Applying Benford s Law to Accounting, 1988 Continuous Process Auditing System, AT&T Bell Laboratories, 1989 Continuous Monitoring Platform Audit Exchange 2.0, 2004 3
Personal Journey - Image courtesy of www.grocerexchange.com 4
Personal Journey Late 90 s / Early 2000 s Cutting Edge Technology DB2 JCL Easytrieve Oracle v 8.0 SQL Server v 6.5 Microsoft Access ACL v 6.5 ACL for MVS Cold Fusion 5
Data Analytics Mission and Team Data Analytics Mission: To be a progressive, collaborative and proactive data analytics function that supports risk identification and monitoring processes, integrated audits, continuous auditing, Division reporting, and proactive fraud reviews and investigations. Data Analytics Team: Tim Penrose, Managing Director, Joined IAD October 2010. Brian Allen, Director. Joined IAD in July 2013. Brian Karp, Manager. Joined IAD in January 2014. Lindsay Holden, Senior Data Analyst. Joined IAD in July 2015. Todd Johnson, Senior Data Analyst. Joined IAD since October 2012. 6
Current DA Tools Diverse and Evolving Toolset: Internal Audit Data Mart Microsoft SQL Server 2012 Visualization Software Tableau Desktop Professional 9.0 and Tableau Server Internal Audit Data Analytics BI Portal SharePoint 2010 Statistical Software (e.g. R and SAS) Big Data Tools Teradata Aster Splunk Desktop Generalized Auditing Software ACL AN 10.5. 7
How do we do this? DA Integration Process Analytic Planning Develop Scripts Update and Maintain Scripts Obtain and Understand Data Analyze and Test Results 8
DA Integration Process - Planning Planning Phase: Scope & Objective Definition Stage Identify and document the scope and risks associated with the engagement and communicate that plan to the audit client. Business Requirements Definition Stage Attend walkthroughs, engage audit partners, and develop a DA test plan that aligns to the process, risks, and controls in Team Mate. Data Acquisition Stage Request and obtain primary and secondary data sets independently, from IT and/or the business. 9
DA Integration Process - Consumption Read / interpret results: Understand what the results tell you and the related risks. Understand the logic that got us there and why we might have false positives. Follow-up procedures: What do we do next? Discuss internally and refine results Include result items in sample testing Follow-up directly with the business to discuss what we are seeing in the data Tips for discussing results with the business: Engage early Proceed with caution Provide timely feedback to DA team to refine analytic / provide lessons learned 10
Consumption Risk Reputational Risk with Business Management and within Audit Division: Results are not properly understood or vetted prior to approaching management, which erodes trust Incorrect conclusions drawn from data Incorrect results in report Audit Risk: Potential exceptions are not identified by DA or are identified by DA and not analyzed/evaluated by Audit Team Risk left on the table 11
What is Data Analytics? Definitions Data analytics is defined as the process of inspecting, cleaning, transforming, and modeling data with the goal of highlighting useful information, suggesting conclusions, and supporting decision making. -Various sources Data analytics is an analytical process by which insights are extracted from operational, financial, and other forms of electronic data internal or external to the organization. These insights can be historical, realtime, or predictive and can also be risk-focused (e.g., controls effectiveness, fraud, waste, abuse, policy/regulatory noncompliance) or performance focused (e.g., increased sales, decreased costs, improved profitability) and frequently provide the how? and why? answers to the initial what? questions frequently found in the information initially extracted from the data. -KPMG 12
Audit Data Analytics Four areas of Audit Data Analytics: Audit and/or Investigation Support Help Desk Incidents example Link Analysis example Self Service Investigation Self Service Dashboard Internal Audit Process (Professional Practices) PPG Dashboard Continuous Auditing / Continuous Monitoring 13
Help Desk Incidents Issue Trigger: Frequent emails sent internally notifying users of Sev 1 and Sev 2 system outages Questions Asked: What is the cause of these issues? Are these issues occurring more frequently than usual? Who is affected by these issues (internal or external customers)? Are these incidents related to a particular line of business? Tool Selection: Tableau Desktop and Server 14
Link Analysis Issue: Device and IP address information was collected from 18 involved participants with confirmed online fraud activity. The data was filtered for known fraudulent indicators. Question Asked: Are these IP addresses and Device IDs connected? If so, what is the relationship between these IP addresses and Device IDs? Tool Selection: Teradata Aster Collaborative Filter function 15
Self Service Issue: During the course of an investigation, our Investigators may need specific pieces of customer information promptly. Question Asked: Can you provide information about a specific customer? Do multiple customers share the same information? Tool Selection: Tableau Desktop and Server 16
PPG Dashboard Issue: Current process of creating and maintaining monthly PPG dashboard is time consuming and cumbersome Question Asked: Can you automate and improve the PPG dashboard process? What additional metrics can be created to monitor audit statuses? Tool Selection: Tableau Desktop and Server 17
Questions or comments? tiaa-cref.org 2011, Teachers Insurance and Annuity Association College Retirement Equities Fund (TIAA-CREF), New York, NY 10017. 18
Brian J. Karp, CIA, CISA, CFE, CRISC Brian.Karp@tiaa-cref.org 704-988-4711