Predictive or retroactive audit slides
Download as PPTX, PDF3 likes1,782 views
The document discusses predictive, preventive, and progressive audits in the context of continuous audit and audit automation. It proposes a conceptual revolution is needed where measurements are full cycle and preemptive. Progressive (P1) audits involve automating actual audit processes by breaking them down into small automated steps. Preventive (P2) audits use forensic models to create preventive filters that flag transactions for review. Predictive (P3) audits use models to predict levels and flows, with variances establishing aberrations. Audit automation and continuous monitoring are seen as complementary and necessary for a real-time economy, but challenges include integrating systems and overcoming organizational obstacles.
Predictive or retroactive audit slides
- 2. Predictive or Retroactive Audit? 25 WCAS CONTECSI 9 Sao Paulo, May 31, 2012 Miklos A. Vasarhelyi KPMG Professor of AIS
- 3. The Present and The Future • Yes -> there is a need for total revamping of business measurement and assurance schemata – The past measurement and assurance compromises and tradeoffs do not work any more under extant information technology – A conceptual revolution is needed, better that is not forced – Measurements that are full cycle and preemptive – Audit Automation: Progressive (P1), preventive (p2) and predictive audits (P3) • A dual frame of standard setting must be put in place to allow for the progressive development and implementation of measurement and assurance in the digital era 3
- 4. Continuous Online Audit Outline • Definition and background • A conceptual revolution • Audit automation and continuous audit • Progressive (P1), preventive (p2) and predictive audits (P3) • Implementation of continuous audit 4
- 6. Continuous Audit Definition • Continuous auditing is a type of auditing which produces audit results simultaneously with, or a short period of time after, the occurrence of relevant events. (CICA/ AICPA 1999) • The Institute of Internal Auditors' (The IIA) defines continuous auditing simply as "any method used to perform audit-related activities on a more continuous or continual basis," without further defining what "more" means. • IIA (GTAG #3 ), continuous auditing is defined as the automatic method used to perform control and risk assessments on a more frequent basis. • ISACA issued guidance in 2010 6
- 7. An evolving continuous audit framework Continuous Risk Monitoring and •Automation Assessment •Sensoring Continuous Continuous Data Control •ERP Audit Monitoring •E-Commerce Continuous Audit 7 7
- 9. Today • Although several surveys state that a substantial percentage of large firms are doing continuous audit this is not at all true • There are several initiatives but manly outside the mainstream process • The real time economy is here but there is substantive lack of progress in the assurance process • Tools and technology exist but the necessary socio- technical factors are not yet here 10
- 10. KPMG Sigma HP Itaú- Bank Unibanco GL PPP Process Verizon KPIs/KRIs CRMA PPP Credit Mining Insuran Card P&G Insuran IDT ce ce Itaú- AT&T A/P UniBanco Inventory Duplicate Payments Dashboard HCA Duratex Siemen Supply Chain CDA A/P CCM s Continuous Met- Control P&G Life Monitoring J+J Talecris Claims American Wires / ACL Inventor CA Technologies FCPA Sales FCPA Water / y Commissio Caseware n Audit Automation Audit Methodologies P&G: Order to Cash •Multidimensional Clustering Auditor Judgment •Process Mining •Continuity Equations Siemens- AAS Automation •Predictive Auditing AICPA – ADS / APS •Visualization •Analytic Playpen
- 11. The old tradeoffs fail with new technology • They are not only inefficient they can be delusory or plain wrong – Sampling vs full population testing – Manual confirmations vs confirmatory extranets (database to database pinging) – Annual audit opinions – Business measurement • Focus on financial numbers (after the horse left the barn) • Accounting rules – LIFO and FIFO » Depreciation » Owners Equity » Goodwill 12
- 12. The old tradeoffs (2) • Must learn to live with a hybrid environment where rules and processes are migrating – Is the FASB able to set standards on an disruptive environment? – Can you change the tires of a car in movement? • Yearly reporting • Assurance • Financial operations – The dire scenario of not changing • False sense of assurance • Progressive loss of value of the financial reporting and auditing functions in exchange to alternate approaches • Substantive societal costs 13
- 13. The role of assurance providers • A much more pervasive presence on the wealth producing landscape – Assurance coordination needs -> the profession did not manage to capitalize on enormous assurance needs in a digital society – Technology validation on a big data / analytic model environment – Boundary validation and interpretation in a multi-source process environment • RER (Real Electronic Reporting) provides a wealth of opportunities that will be fulfilled by someone/ something – Goodbye bill by hours /economic model of the profession – Goodbye yearly audit / continuous assurance – Goodbye manual audits – Goodbye retroactive audit / predictive/ preventive audits 14
- 14. Electronic measurement and reporting (XBRL) • XBRL although a very positive step on the route towards automation perpetuates some of the weaknesses of the “paper oriented” reporting model – Audits to improve their social agency function should be of corporate measurement and databases not of financial reports – As most substantive regulatory based changes XBRL presents a series of unintended consequences including • Pressure toward standardization of reporting • Facilitation of more frequent reporting • Evolutionary force towards the standardization of the semantics of accounting reporting • A poor conduit to represent corporate transactions (XBRL/FR) • XBRL/FR will eventually lead to XBRL/GL –great societal effects 15
- 15. CONCEPTUAL R->EVOLUTION 16
- 16. Where the conceptual revolution lays • A forward looking audit (predictive) 1 – Models predict levels and flows – Variances establish aberrations – Evidence is weighted and evaluated – New forms of evidence arise – Preventive prediction models are also to be used – BUT 17
- 17. Conceptual revolution (2) • 2 Retroactive (but for very recent period) is still needed – Models cannot capture the unexpected – PCAOB requirements are not only anachronistic but counterproductive and expensive – There is a natural interlinkage between CDA / CCM / and CRMA; they are complementary and related – ALSO 18
- 18. Conceptual revolution (3) • 3 Control Monitoring has to be Automated – Controls in ERPs are not observable and they are user configurable – PCAOB requirements are not only anachronistic but counterproductive and expensive – There is a natural inter-linkage between CDA / CCM / and CRMA; they are complementary and related • E.g. although controls may be active and effective there is never certainty that all risk are covered and that no new form of fraud has been invented – ALSO 19
- 19. Conceptual revolution (4) • 4 CRMA – Bear Sterns collapsed weeks after a clean audit opinion – Auditors stated that conditions changed dramatically in a short period of time – PCAOB has been pressuring for a “risk based audit” but this has not been clearly specified and is held back by ridiculous (for example sampling) requirement 20
- 20. AUDIT AUTOMATION AND CONTINUOUS AUDIT 21
- 21. • There is no real time economy audits if at least parts of the assurance process is not automated • The major obstacle that internal auditors face is the availability and access to data • The second major obstacle is the multiplicity of audit-like organizations with splintered needs and objectives • Audit automation modules have to ultimately be built-in to production processes and cooperate with these although having different lords (owners) • They have to closely interact with analytic models and human interaction modules 22
- 22. PROGRESSIVE, PREVENTIVE AND PREDICTIVE AUDITS 23
- 23. Continuous Audit and Audit Automation (P1 P2 P3) • The progressive audit (P1) – Where actual audit processes are formalized / broken down into small steps and parts automated – Coherent with the proposed ASEC – Audit Data Standards – Teeter (2013) automates steps in the Siemens and P&G audits through breaking them down in Audit Actions and then creating automation for them • The location of the auditor • The procedure adopted • The timing / frequency of the procedure 24
- 24. Audit Data Standards & Apps AICPA’s Assurance Services Executive Committee – June 2011
- 25. Relationship Between Audit Apps and Common Data Data acquisition Data access Audit apps (based on assertions) Common Dashboard Production Data data Repository Analytic Query Trend Data Ratio matching Query Dashboard Black box Activity log Classify logs ERP vendors Cloud/data providers Platform developers
- 26. Introduction Data & Procedures Dimensions of data/procedures Framework Conclusion • Method – Data generation – Audit procedure • Timing Automated – Data generation – Data frequency – Audit frequency • Location Manual – Data storage/access – Audit steps Discrete Continuous
- 27. Dimensions of Assurance • Automation Retroactive -> Predictive – Manual vs Automated • Timing – Discrete vs continuous • Location – Local vs remote • Focus Automated – retroactive vs predictive • Procedure Procedures – Confirmation – Physical verification – Aging of receivables Manual – Cutoffs – Etc Discrete Continuous
- 28. Audit Focus Retroactive Preventive Predictive Not Preventive 29
- 29. Continuous Audit and Audit Automation (P1 P2 P3) • The preventive audit (P2) – Where based on forensic models preventive (and hopefully adaptive) filters are created and are placed in the actual process preventing transactions with high discriminant loadings to be processed and deflecting these to a review process – This is a preventive control / associated with a review and analytic audit process – See Yong Bum Kim’s dissertation (2011) for basic work on the approach 30
- 30. YONG BUM KIM INCORPORATING FORENSIC INTO AN CA/CM PHILOSOPHY 31
- 31. Incorporating Forensics into an CA/CM philosophy Operations Forensic archives Forensic models models controls Forensic analysis Filtering by Audit Forensic By models Exception 32
- 32. – – – 33
- 35. 36
- 36. 37
Editor's Notes
- #7: One thing is clear: there is no clear and precise definition of ‘Continuous Audit’. One thing is apparent in all these and that is that ‘continuous audit’ is held to be an automated approach. Instead of using samples taken at various time intervals, 100% of the data population can be tested continually.
- #12: M
- #36: Accuracy = (TP + TN) / NSpecificity = True negative rate (how many % of cancellation transactions that the model can correctly predict as cancelled.)Recall = the proportion of Real Positive cases that are correctly Predicted Positive. (how many % of non -cancellation transactions that the model can correctly predict as non-cancel.)Precision = the proportion of Predicted Positive cases that are correctly Positives. (TP/ (TP+FP))False alarm rate = focus on FP (predict cancel as non-cancel)