Audit analytics and the agile auditor
1 like326 views
The document provides an overview of audit analytics and outlines key steps for starting an analytics project, emphasizing the importance of data familiarity and iterative feedback. It showcases auditnet® as a significant resource for auditors, offering templates, webinars, and guides focused on data analytics and internal audit practices. Additionally, it highlights various data analysis techniques and software tools for efficient auditing, along with essential information regarding Continuing Professional Education (CPE) for webinar participants.
Audit analytics and the agile auditor
- 1. 7/23/2018 1 Audit Analytics and the Agile Auditor: Starting an Analytics Project Deniz Appelbaum, PhD, MBA Assistant Professor of Accounting and Finance, Montclair State University About Jim Kaplan, CIA, CFE Page 2 President and Founder of AuditNet®, the global resource for auditors (available on iOS, Android and Windows devices) Auditor, Web Site Guru Internet for Auditors Pioneer IIA Bradford Cadmus Memorial Award Recipient Local Government Auditor’s Lifetime Award Author of “The Auditor’s Guide to Internet Resources” 2nd Edition
- 2. 7/23/2018 2 AuditNet®, the global resource for auditors, serves the global audit community as the primary resource for Web-based auditing content. As the first online audit portal, AuditNet® has been at the forefront of websites dedicated to promoting the use of audit technology. Available on the Web, iPad, iPhone, Windows and Android devices and features: • Over 2,900 Reusable Templates, Audit Programs, Questionnaires, and Control Matrices • Webinars focusing on fraud, data analytics, IT audit and internal audit with free CPE for subscribers and site license users • Audit guides, manuals, and books on audit basics and using audit technology • LinkedIn Networking Groups • Monthly Newsletters with Expert Guest Columnists • Surveys on timely topics for internal auditors Page 3 About AuditNet® LLC HOUSEKEEPING • This webinar and its material are the property of AuditNet® and its Webinar partners. Unauthorized usage or recording of this webinar or any of its material is strictly forbidden. • If you logged in with another individual’s confirmation email you will not receive CPE as the confirmation login is linked to a specific individual. • This Webinar is not eligible for viewing in a group setting. You must be logged in with your unique join link. • We are recording the webinar and you will be provided access to that recording after the webinar. • Downloading or otherwise duplicating the webinar recording is expressly prohibited. • If you have indicated you would like CPE you must attend the entire Webinar to receive CPE (no partial CPE will be awarded). • If you meet the criteria for earning CPE you will receive a link via email to download your certificate. The official email for CPE will be issued via NoReply@gensend.io and it is important to white list this address. It is from this email that your CPE credit will be sent. There is a processing fee to have your CPE credit regenerated post event. • Submit questions via the chat box on your screen and we will answer them either during or at the conclusion. • You must answer the survey questions after the Webinar or before downloading your certificate.
- 3. 7/23/2018 3 IMPORTANT INFORMATION ABOUT CPE! SUBSCRIBERS/SITE LICENSE USERS - If you attend the entire Webinar you will receive an email with the link to download your CPE certificate. The official email for CPE will be issued via NoReply@gensend.io and it is important to white list this address. It is from this email that your CPE credit will be sent. There is a processing fee to have your CPE credit regenerated post event. NON-SUBSCRIBERS/NON-SITE LICENSE USERS - If you attend the entire Webinar and requested CPE you must pay a fee to receive your CPE. No exceptions! We cannot manually generate a CPE certificate as these are handled by our 3rd party provider. We highly recommend that you work with your IT department to identify and correct any email delivery issues prior to attending the Webinar. Issues would include blocks or spam filters in your email system or a firewall that will redirect or not allow delivery of this email from Gensend.io. Anyone may register, attend and view the Webinar without fees if they opted out of receiving CPE. We are not responsible for any connection, audio or other computer related issues. You must have pop-ups enabled on you computer otherwise you will not be able to answer the polling questions which occur approximately every 20 minutes. We suggest that if you have any pressing issues to see to that you do so immediately after a polling question. The views expressed by the presenters do not necessarily represent the views, positions, or opinions of AuditNet® LLC. These materials, and the oral presentation accompanying them, are for educational purposes only and do not constitute accounting or legal advice or create an accountant-client relationship. While AuditNet® makes every effort to ensure information is accurate and complete, AuditNet® makes no representations, guarantees, or warranties as to the accuracy or completeness of the information provided via this presentation. AuditNet® specifically disclaims all liability for any claims or damages that may result from the information contained in this presentation, including any websites maintained by third parties and linked to the AuditNet® website. Any mention of commercial products is for information only; it does not imply recommendation or endorsement by AuditNet® LLC
- 4. 7/23/2018 4 Audit Analytics and the Agile Auditor: Starting an Analytics Project • Introduction • Project Planning • Know Your Subject • Know the Data • Run Your Analysis • Obtain Feedback • Re-run Your Analysis • Share Results • Parting Thoughts AGENDA
- 5. 7/23/2018 5 • Introduction • Project Planning • Know Your Subject • Know the Data • Run Your Analysis • Obtain Feedback • Re-run Your Analysis • Share Results • Parting Thoughts AGENDA The New Agile Auditor
- 6. 7/23/2018 6 Data Analytics: What does it actually mean? Data Analytics - A process of inspecting, cleaning, transforming, and modelling data with the goal of highlighting useful information, suggesting conclusions, and supporting decision making Data Mining is a particular data analysis technique that focuses on modelling and knowledge discovery for predictive rather than purely descriptive purposes Business Intelligence - Covers data analysis that relies heavily on aggregation (summarization), focusing on business information Predictive Analytics focuses on application of statistical or structural models for predictive forecasting or classification Text Analytics - Applies statistical, linguistic, and structural techniques to extract and classify information from textual sources, a species of unstructured data • Introduction • Project Planning • Know Your Subject • Know the Data • Run Your Analysis • Obtain Feedback • Re-run Your Analysis • Share Results • Parting Thoughts AGENDA
- 7. 7/23/2018 7 BASIC AUDIT ANALYTICS PROJECT FORMAT Data and Preliminary Project Analysis • Familiarity with data and project requirements • Analysis of texts and documents; exploratory tests Data and Preliminary Project Analysis • Familiarity with data and project requirements • Analysis of texts and documents; exploratory tests Unstructured Interviews and Observations • Gains more knowledge from the subjects involved • Yields enough information for first pass test Unstructured Interviews and Observations • Gains more knowledge from the subjects involved • Yields enough information for first pass test Preparation of First Pass Test • Beginnings of project file • Continual refinement of analytical processes • Runs concurrent to manual processes Preparation of First Pass Test • Beginnings of project file • Continual refinement of analytical processes • Runs concurrent to manual processes More Interviews and Observations • Analysis of First Pass test • Unstructured and Structured • Yields enough information for second pass tests More Interviews and Observations • Analysis of First Pass test • Unstructured and Structured • Yields enough information for second pass tests Preparation of Second Pass Tests • Refinement of the test file • Results of the analytics compared to those of the control, the manual process Preparation of Second Pass Tests • Refinement of the test file • Results of the analytics compared to those of the control, the manual process Special Tasks • Yields continued refinements to the system, continuous methodology • Ongoing refinements to process and procedures as needed Special Tasks • Yields continued refinements to the system, continuous methodology • Ongoing refinements to process and procedures as needed • Introduction • Project Planning • Know Your Subject • Know the Data • Run Your Analysis • Obtain Feedback • Re-run Your Analysis • Share Results • Parting Thoughts AGENDA
- 8. 7/23/2018 8 • Interviews • Observations • Online Searches • Understand the objectives and assertions • Iterative inquiries • Feedback KNOW YOUR SUBJECT • Introduction • Project Planning • Know Your Subject • Know the Data • Run Your Analysis • Obtain Feedback • Re-run Your Analysis • Share Results • Parting Thoughts AGENDA
- 9. 7/23/2018 9 Data Sources and Extractions Big Data: data of high volume, high velocity, and high variety that requires new and different forms of processing to enable enhanced decision making, insight discovery, and process optimization • Continuous information source • Qualitative data • PDF documents • Twitter/Blog feeds • Audio and video files • Emails, texts, corporate minutes • Mainframe and laptop software and logs • GPS data • Phone call meta-data • Receipts • Interview recordings/logs Preparation Phase Identify relevant data Obtain the data Verify the data • Control totals • Correct periods • Gaps/missing fields • Reasonableness tests • Duplicates Cleanse/Normalize Data
- 10. 7/23/2018 10 POLLING QUESTION 1 • Introduction • Project Planning • Know Your Subject • Know the Data • Run Your Analysis • Obtain Feedback • Re-run Your Analysis • Share Results • Parting Thoughts AGENDA
- 11. 7/23/2018 11 Five Advantages of Using Data Analysis Software 1) Allows an auditor to centralize an investigation 2) Assures completion and accuracy 3) Bases predictions about the probability of a fraudulent situation on reliable statistical information 4) Allows searches of entire data files for red flags of possible errors, misstatements, and control issues 5) Assists in the development of reference files for ongoing audit projects and work files Possible Data Mining and Analysis Software Excel CaseWare IDEA ACL ActiveData for Excel Thompson Reuters Tableau Python R WEKA Oversight SAS Oracle IBM Watson and IBM Blockchain
- 12. 7/23/2018 12 Core Data Analysis Functions in Software Packages • Sorting • Record Selection • Joining Files • Multi-file processing • Correlation Analysis • Trend Analysis • Time Series • Verifying multiples of a number • Compliance verification • Duplicate searches • Expressions and Equations • Graphing • Filter and Display criteria • Fuzzy logic matching • Gap tests • Pivot tables • Regression Analysis • Sort and index • Statistical analysis • Stratification • Date functions • Benford’s Law analysis Sorting Arrange the data in a meaningful order for analysis
- 13. 7/23/2018 13 Record Selection • Select specific records for analysis • For example: NYC Office of the Mayor employees with OT pay in 2016 Joining Files • Connects fields from two sorted input files into a third file • Frequently used to match invoice data with A/R files, using common identifier
- 14. 7/23/2018 14 Multi-File Processing • Allows the user to relate several files by defining their relationship without the use of join • For example, relate an outstanding invoice master file to A/P file using an account number. Can relate invoice numbers as well Correlation Analysis • Relationships in raw data • Examine correlations in data for deviations from expected relationships • Pair-wise relationship between two sets of data; each x has a unique y • The strength of this relationship is measured by the correlation co-efficient • In excel, the CORREL(array1,array2) function returns this coefficient
- 15. 7/23/2018 15 Trend Analysis • Calculates the values of data over time and forecasts values into the future based on the assumption that the expected behavior will continue • Beneficial for auditors to benchmark future behaviors of accounts, persons, transactions types • Seasonal data should be examined with Time Series Analysis • Quantifies the trend of the data – which department shows a supplies expense that exceeds past trends? Time Series Analysis Calculates the trend of data over time with a seasonal component Decomposition Method of Time Series Analysis is the most useful for FINANCIAL data Testing based on seasonality – higher values at year end?
- 16. 7/23/2018 16 Verifying Multiples of a Number • Are numbers consistent with the regular or expected rate? • Or, are transactions under or above the limit? • Or, do they lie just below the limit? • IDEA limits tests/IDEA stratifications Duplicate Searches Duplicate Searches Identify duplicate values in specified fields Single file or joined files Addresses, identifiers, days, amount
- 17. 7/23/2018 17 Expressions and Equations • Auditors can build expressions and equations based on their knowledge and expectations of the data • Also used with compliance testing Filter and Display Criteria • Auditors can create filters or queries based on specific user-defined criteria that results in only those records being displayed • Can be deployed when loading the data or later as an extraction or criterion for another test • In IDEA: Equation Editor box
- 18. 7/23/2018 18 Fuzzy Logic Matching • Matching very similar attributes that might escape normal matching algorithms • For example: First Street, First St, and 1st Street • Very useful when the perpetrator has taken steps to mask steps • May produce an increased number of false positives Gap Tests • Identifies items missing in expected sequences or series (check and invoice numbers) • Finds sequences where none are expected to exist (employee government ID numbers, SSNs)
- 19. 7/23/2018 19 Pivot Tables • Interactive data summarization tool found in Excel and in IDEA • Used to sort, count, total, or give the average of specified data • Provides the “Big Picture” Regression Analysis Statistical method that uses a series of records to create a model relationship between a dependent variable and one or more independent variables Ex: Regression could be used to determine the number of widgets manufactured based on materials and labor numbers Periods where sales of widgets are higher or lower than expected would require analysis
- 20. 7/23/2018 20 Sort and Index • Arranges the data in a manner that assists analysis – ascending, descending • Depending on the field type, could be alphabetically or numerically Statistical Analysis/Descriptive Statistics • Calculating statistics such as averages, mins and maxes and absolute values • IDEA Field Statistics
- 21. 7/23/2018 21 Stratification • Breaks the data down into intervals or strata • Very useful for limits testing! Date Functions Aging analysis
- 22. 7/23/2018 22 POLLING QUESTION 2 Benford’s Test • Founded on counterintuitive observation that individual digits of multi-digit numbers are not random, but follow a pattern • Describes expected frequencies of digits in numbers • UNTAMPERED NATURALLY OCCURING NUMBERS! • Posits that distribution of first digits is positively skewed, or more heavily weighted toward smaller numbers • Number series must follow a geometric sequence • Each successive number calculated as a fixed percentage increase over previous number Applications • Net income • Earnings per share • Income tax • Fraud detection
- 23. 7/23/2018 23 Data Analysis Techniques and Their Required Data Types Benford’s Test: Expected Digital Frequencies First Digit Test • Compares the first-digit profile of a data set to Benford’s first digit profile
- 24. 7/23/2018 24 First-Two-Digits Test • Compares the first two digits of a data set with Benford’s profile for the first two digits (purchases at $300 threshold) Last-Two-Digits Test • Compares the last two digits of a data set with Benford’s profile for the last two digits
- 25. 7/23/2018 25 Data Analysis Techniques and Their Required Data Types Technique Type: E or C S, SS, U QN, QL D, S Transaction Tests, Ratio Analysis C S QN D Sampling C S QN S Confirmations/Re-performance C S QN D Clustering Models E S QN S Text Mining Models E SS, U QL S Visualizations E SS, U QL, QN S Artificial Neural Networks (ANN) C S QN S Expert Systems/Decision Aids C S, SS, U QN, QL S C4.5 statistical classifiers C S QN S Bagging and Boosting Models C S QN S Bayesian Theory/Bayesian Belief Networks (BBN) C S QN S Probability Theory Models C S QN S Log Regression C S QN S Linear Regression C S QN S Time Series Regression C S QN S Univariate and Multivariate Regression Analysis C S QN S Benford’s Law C S QN S Descriptive Statistics E S QN S E, C = Exploratory, Confirmatory; S, SS, U = Structured, Semi-Structured, Unstructured; QN, QL = Quantitative, Qualitative; and D, S = Deterministic, Statistical (Adapted fromAppelbaum et al 2017) • Introduction • Project Planning • Know Your Subject • Know the Data • Run Your Analysis • Obtain Feedback • Re-run Your Analysis • Share Results • Parting Thoughts AGENDA
- 26. 7/23/2018 26 Post- Analysis Phase • Respond to Analysis Findings • Monitor the Data Spectrum of Analysis: • Ad-hoc testing • Repetitive testing • Continuous testing • Introduction • Project Planning • Know Your Subject • Know the Data • Run Your Analysis • Obtain Feedback • Re-run Your Analysis • Share Results • Parting Thoughts AGENDA
- 27. 7/23/2018 27 • Introduction • Project Planning • Know Your Subject • Know the Data • Run Your Analysis • Obtain Feedback • Re-run Your Analysis • Share Results • Parting Thoughts AGENDA Graphics • Invaluable for exploratory and explanatory analysis • Graphics are more exciting than spread sheets! Graphical Pivot Table!
- 28. 7/23/2018 28 Payroll/Employee Expense Case Studies Case Study: Payroll Fraud – Test For Employees with the Same Address
- 29. 7/23/2018 29 Case Study: Payroll Fraud – Result Expense Reimbursement Schemes Travel and Entertainment expenses Procurement Cards • Personal items • Expenses that never materialized or were subsequently canceled • Fake or Altered Receipts Improper Expense claims include: • Overstated Expense Reimbursements • Mischaracterized Expense Reimbursements • Multiple Reimbursements • Fictitious Expense Reimbursements Fraudulent Expense reimbursements:
- 30. 7/23/2018 30 Falsified Travel Expenses Load the provided excel travel expenses file into IDEA as a managed project called Employee Travel Expenses Days Traveled Test: Create a field called DATE_DIFF with the equation: @Age(END_DATE,START_DATE) Same Day Traveled with Accommodation Charges: DATE_DIFF = = 0 .AND. ACCOMMODATION> 0 Same Day Traveled with Flight Charges: DATE_DIFF = = 0 .AND. AIR_FARE> 0 Same Day traveled with both fight and accommodation changes: DATE_DIFF = 0 .AND. AIR_FARE > 0 .AND. ACCOMMODATION> 0
- 31. 7/23/2018 31 Travel Expenses Traveled with Flight but No Accommodation Charges: DATE_DIFF > 0 .AND. AIR_FARE > 0 .AND. ACCOMMODATION = 0 SAME-SAME-SAME (Duplicate Tests) Using the main Travel Expenses data set, test for duplicates on START_DATE and for EMPLOYEE_NO. Using the main Travel Expenses data set, test for duplicates on EMPLOYEE_NO and AIR_FARE, where AIR_FARE is greater than 0. Likewise, test for duplicates on EMPLOYEE_NO and ACCOMMODATION, where ACCOMMODATION > 0. Extraction bases on Audit Unit: Please extract the following, creating a new data set called Ass.Dep.Min. POSITION = = “Assistant Deputy Minister”. Please display the result in a graphic format that you feel is most appropriate. Employee Procurement Cards Measure for Jan 2013 to April 2014 Total Data Set Missing Purchase Item Information Data Set # of Transactions 741,710 194,528 (26% of total) # of Employee IDs 4532 (cards are 5600) 4339 Total $ Fin Original Currency $157,115,184 $65,926,544 (42% of total) Total # of vendors 101,900 41,258 Merchant # of Trans # Emp ID $ Total # of ?? Walmart 4171 1290 $343,750 All Sam’s Club 819 259 $126,612 All Amazon 11,690 276 $19,302 Non-credit Target 224 115 $37,170 All Ulta/Sally B 51 21 $6804 15 (29%) Petsmart 174 43 $12,328 25 (14%) PetCo 116 9 $60,764 none ? Another informative merchant: PETSMART 174 transactions by 43 cards for $12,328
- 32. 7/23/2018 32 Employee Procurement Cards Employee Procurement Cards – A Few Examples
- 33. 7/23/2018 33 Employee Procurement Cards – Interactive Tool POLLING QUESTION 3
- 34. 7/23/2018 34 • Introduction • Project Planning • Know Your Subject • Know the Data • Run Your Analysis • Obtain Feedback • Re-run Your Analysis • Share Results • Parting Thoughts AGENDA Additional Visual Analytics: Tree Maps Graphical representation of the data by rectangular spaces, size, and colors/intensities
- 35. 7/23/2018 35 Additional Visual Analyics: Link Analysis with SAS Visual Investigator • Creates visual representations of links between people, social networks, and indirect relationships • Beneficial for auditors: can track the placement, layering, and integration of money as it moves around unexpected sources • Can also: associate communications, uncover indirect relationships, show discreet connections, and demonstrate complex networks Geo-Spatial Analysis Displays geo-locational data along with other attributes, can reveal intersections and clusters of behavior
- 36. 7/23/2018 36 Clustering (with WEKA) As defined by Sharma & Panigrahi (2013): “is known as gaining insights and identifying interesting patterns from the data stored in large databases in such a way that the patterns and insights are statistically reliable, previously unknown, and actionable “. Cluster analysis as a data mining technique helps finding similar objects in data. Kaufman & Rousseeuw (2009) have defined clusteranalysis as ”theart of findinggroups in data.” Legacy costs and budget maneuvers explain 58.42% of the point variability...
- 37. 7/23/2018 37 Artificial Intelligence and the Auditor • Artificial intelligence (AI) allows IT systems to imitate the cognitive ability of human – “problem solving”, “reasoning”, “planning” and “learning” • AI enabled systems possess inbuilt intelligence to sift through, aggregate, blend, and identify patterns and relationships that are buried within mountains of data – a large number and types of data sources For Example: • Customer onboarding • Link analysis • Customer segmentation • Screening • Risk management • Transaction monitoring • Alert investigation, reporting and case management Clustering (with WEKA) Evaluating Data Analysis Software Data import/export capabilities Data visualization Suite of tools? Tailoring: Performance Functionality Usability Support for additions
- 38. 7/23/2018 38 Remember: Trust but Verify! Thank You Deniz Appelbaum appelbaumd@Montclair.edu