Aasmund (2020).pdf

See discussions, stats, and author profiles for this publication at: https://www.researchgate.net/publication/342140939
An Exploratory Study into the Use of Audit Data Analytics on Audit Engagements
Article in Accounting Horizons · June 2020
DOI: 10.2308/HORIZONS-19-121
CITATIONS
23
READS
3,828
4 authors, including:
Aasmund Eilifsen
NHH Norwegian School of Economics
29 PUBLICATIONS 1,059 CITATIONS
SEE PROFILE
Finn Kinserdal
4 PUBLICATIONS 35 CITATIONS
SEE PROFILE
William F. Messier Jr
86 PUBLICATIONS 3,135 CITATIONS
SEE PROFILE
All content following this page was uploaded by Finn Kinserdal on 17 June 2020.
The user has requested enhancement of the downloaded file.

Downloaded
from
http://meridian.allenpress.com/accounting-horizons/article-pdf/doi/10.2308/HORIZONS-19-121/2509108/acch-horizons-19-121.pdf
by
Aasmund
Eilifsen
on
15
June
2020

Aasmund Eilifsen*
aasmund.eilifsen@nhh.no
Finn Kinserdal
finn.kinserdal@nhh.no
William F. Messier, Jr.
william.messier@nhh.no
Thomas E. McKee
The Medical University of South Carolina and NHH Norwegian School of Economics
temckee@musc.edu
* Corresponding author.
August 2019
Revised: February 2020
Accepted: May 2020
Running Head: Use of Audit Data Analytics
Downloaded
from
by
Aasmund
Eilifsen
on
15
June
2020

2
Acknowledgments: We would like to thank all of the participating partners, managers, and their
firms for their assistance with this research. We also thank the participants at the Norwegian
Institute of Public Accountants (DnR) and NHH Roundtable: Perspectives on Digitalization of
the Audit Process, workshop Vrije Universiteit Amsterdam, 2019 European Auditing Research
Network (EARNet) Symposium, our discussant Arpine Maghakyan, 2019 Accounting Horizons
and Baruch College Conference on Data Analytics in Accounting, our discussant Travis Holt,
and two anonymous reviewers for their helpful comments. This research is supported by a grant
from the Research Council of Norway. Professor Messier also received financial support for this
research from his Norwegian Institute of Public Accountants (DnR) Professorship in Auditing
Professorship in Auditing at NHH.
Downloaded
from
by
Aasmund
Eilifsen
on
15
June
2020

ABSTRACT
This study explores the use of audit data analytics (ADA) in current audit practice. First, we
interviewed the heads of professional practice of five international public accounting firms in
Norway. We find that they differ in strategies on how to implement ADA and the heads report
significant uncertainty about the supervisory inspection authorities’ response to the use of ADA.
Second, we administered a questionnaire to 216 engagement partners and managers about their
perceptions of ADA as well as their actual ADA use on 109 audit engagements. Overall, the
attitudes towards ADA usefulness are positive. Analysis of the audit engagements suggests use
of ADA is relatively limited and use of more “advanced” ADA is rare. More ADA are used for
clients with integrated ERP/IT systems and for newly tendered audit engagements. We also
provide details of actual ADA use on each phase of the audit. We discuss our findings from an
institutional theory perspective.
Data Availability: The data used in this study is confidential by agreement with the participants.
Keywords: Audit data analytics (ADA); audit engagements; auditors’ perceptions about ADA;
actual use of ADA.
Downloaded
from
by
Aasmund
Eilifsen
on
15
June
2020

2
“Analytics is revolutionizing the audit and facilitating the introduction of disruptive
technologies. Audit is undergoing a data revolution. There has been rapid expansion in
the types and volumes of data that companies produce, providing richer sources of
information to be used when conducting an audit. Digital analytics tools are enabling
industry professionals to look at businesses in a different way by analyzing these bigger
data populations. This provides a deeper understanding of risk, performance and
opportunities.” Hermann Sidhu, EY Global Assurance Digital Leader (EY 2017a).
I. INTRODUCTION
The objective of this study is to explore the use of audit data analytics (ADA) in current
audit practice. Recently, the major international public accounting firms have invested heavily to
advance their audit technology and moved to leverage the use of ADA for financial statement
audits (Deloitte 2016; KPMG 2016; PwC 2017; EY 2017b). ADA are defined as “the science
and art of discovering and analyzing patterns, identifying anomalies, and extracting other useful
information in data underlying or related to the subject matter of an audit through analysis,
modeling, and visualization for the purpose of planning or performing the audit” (AICPA 2014).1
Proponents of the use of ADA cite three potential benefits from their use: (1) improved
understanding of an entity’s operations and associated risks, including the risk of fraud, (2)
increased potential for detecting material misstatements, and (3) improved communications with
those charged with governance of audited entities (AICPA 2017). While a number of researchers
have provided insight into the potential application of various ADA tools on the audit (for an
1
ADA are sometimes intertwined with the term “Big Data.” Gartner (2019) defines Big Data as “high-volume, high-
velocity, and high-variety information assets that demand cost effective, innovative forms of information processing
for enhanced insight and decision making.” Thus, Big Data extends the domain of data from financial data to non-
financial data, from structured to unstructured data, and from inside the organization to outside it (AICPA 2015).
Our primary focus is on the use of ADA; i.e., analytical audit tools and techniques that may, or may not, use Big
Data. This focus is supported by Walker and Brown-Liburd (2019) who find external auditors’ definitions of data
analytics most aligned with the AICPA’s definition.
Downloaded
from
by
Aasmund
Eilifsen
on
15
June
2020

3
overview see Griffin and Wright 2015; Appelbaum, Kogan, and Vasarhelyi 2017, 2018),2
the
availability and the actual use of ADA in audit practice remain largely unknown.
We apply a two-stage research approach.3
First, we interviewed the heads of professional
practice (“Heads”) of five international public accounting firms in Norway4
to get an
understanding of the status of ADA in each firm. Second, we obtained responses to a detailed
questionnaire from a large sample of 216 partners and managers who were in charge of 109 audit
engagements about (1) their perceptions and insights on issues related to ADA and (2) extensive
information on the actual use of ADA in those 109 engagements.5
Our results show the following. First, the Heads indicated that their firms do not require
mandatory use of “advanced” ADA tools. This seems to be driven by the uncertainty about how
the supervisory inspection authorities will evaluate and accept ADA generated audit evidence.
Second, while ADA use is high on the firms’ agenda and there is a global firm push for ADA to
be used on audit engagements, actual use is limited in our sample, including even the audits that
were identified by the Heads as expected to use ADA. Third, based on our discussions with the
Heads, the firms differ in their strategies in how they implement the use of ADA in their
organizations from, at one end of the spectrum, “wait and see” approach to, at the other end of
2
Salijeni, Samsonova-Tadderu, and Turley (2019) observe that to date research has devoted most attention to
potential impact of data analytics on auditing conceptually. Likewise, Austin, Carpenter, Christ, and Nielson (2019,
4) state that “… the existing literature on data analytics consists primarily of thought pieces.” Walker and Brown-
Liburd (2019, 2) observe: “Other than Austin et al. (2018), the literature addresses the potential of data analytics, as
opposed to whether and how data analytics is used in the audit process.”
3
Under a four-year research project at the Norwegian School of Economics (NHH) funded by the Research Council
of Norway on digital transformation of the audit process (Digital Audit Research Project 2020), the five largest
international public accounting firms in Norway contracted to provide data and subjects for the research project.
4
Similar to the U.S., high investor protection and strong legal enforcement characterize the Norwegian financial
reporting and auditing environment (e.g., Leuz, Nanda, and Wysocki 2003; Choi and Wong 2007; Hope, Kang,
Thomas, and Yoo 2009). In addition, Norway is rated high in digital competitiveness (EC 2018) and where auditor
public oversight is considered less restrictive than in the U.S. Alles (2015) and Appelbaum et al. (2017) argue that
public oversight may be an impediment to the adoption of ADA.
5
For simplicity, we refer to the participants as partners and managers. However, some participants held different
positions within the firm. See Table 1 for a detailed distribution of participants.
Downloaded
from
by
Aasmund
Eilifsen
on
15
June
2020

4
the spectrum, centralization of ADA functions and significant firm involvement to facilitate
ADA use. Fourth, the partners and managers indicated that their knowledge and training with
firm available ADA tools were sufficient to permit their use of ADA and their attitudes towards
ADA usefulness are more positive for firm audits in general than for the sampled audit
engagements. Fifth, as expected more ADA are used on engagements where the client has an
integrated ERP/IT system. Sixth, there is a higher frequency of ADA use on new audit
engagements. Participants indicated that recent tender offers specifically asked about firms’ use
of new technology and ADA in audits, and that the audit firms promoted ADA use in the tender
process.6
Seventh, we identify the various phases of the audit where ADA are used. In the audit
planning phase, ADA are used for overall assessment of the client's operations and performance,
identifying and assessing key risks, and mapping of different business processes. In the
substantive testing phase, ADA are used for journal entry testing, calculating sample-size,
selection of random samples, and summarizing ledgers. In the completion phase of the audit,
ADA are primarily used for reconciliation and control between final accounts and underlying
ledgers, analytical procedures, and final review of financial statements. However, overall, the use
of ADA in each phase of the audit is low and there is little use of what would be considered
advanced ADA (i.e., statistical regressions, clustering techniques, statistical predictive analysis,
computerized process-mapping, etc.). The use of Big Data and text-mining is almost non-
existent. When ADA are used, ADA output is mostly used as supplementary evidence. In
summary, our results suggest that the use of ADA within the firms is limited and at an early
stage of implementation. Lastly, we believe that institutional theory provides a framework for
6
After completing the questionnaire, open (semi-structured) discussions took place where participants were offered
the opportunity to comment on the survey and otherwise express any views related to the study. In two of these firm
sessions ADA usage in the tender process was mentioned.
Downloaded
from
by
Aasmund
Eilifsen
on
15
June
2020

5
explaining our findings. Institutional theory suggests that the limited use of ADA will persist
until ADA usage is proved to be superior to the current evidence gathering process for audit
team leaders and their use is supported by the firms, regulators, and supervisors.7
This research makes a number of important contributions. First, little empirical research
exists about ADA since they are relatively new technologies and gaining access to auditors is
challenging (Austin et al. 2019). Gepp, Linnenluecke, O'Neill, and Smith (2018, 110) argue that
qualitative, interview-based studies are needed to fill this knowledge gap. Our study provides
insight into how firms’ leadership and engagement partners and managers perceive the prospects
and impediments to ADA use. Second, we are the first study to systematically document the
actual usage of ADA on current audit engagements. Auditors’ actual use of ADA reflects
auditors’ judgments and decisions about the efficiency and effectiveness of ADA use. Third, we
also provide information on the audit areas where ADA are being used (or not used) as an
evidential source (i.e., sufficient and appropriate audit evidence) and the obstacles that impede
their use. Fourth, we discuss our findings from the perspectives of institutional theory to better
understand auditors’ complex decision of ADA usage and the observed limited use of ADA in
our sample. We suggest that the limited use of ADA will likely persist until its use is
incorporated internally in the firms’ methodology and reward systems; explicitly supported and
accepted by standard-setters and supervisory bodies; and practitioners must find that ADA use
proves efficient and effective in the evidence gathering process. Finally, our research provides a
starting point for practitioners and researchers in validating the efficiency and effectiveness of
the use (or non-use) of ADA. In sum, our study documents the current status of auditors’ use of
7
Theoretical perspectives are conceived as cones of light directed on a phenomenon which exert influence (1) on the
framing of research questions, and (2) guide data collection and analysis (O’Dwyer 2010). Given our ex ante beliefs
about the use of ADA in current audit practice we found it premature to develop research questions.
Downloaded
from
by
Aasmund
Eilifsen
on
15
June
2020

6
ADA at the engagement level and develops an understanding of why ADA use has not yet
fulfilled its promised potential.
The reminder of the paper is organized as follows. The next section provides the
background and overview for the study. This is followed by a section that describes the first
stage of the research — interviews with heads of professional practice. The next section presents
the second stage of the research — a questionnaire about auditor perceptions of ADA and actual
use of ADA at the engagement level. This is followed by a section that discusses our findings
within perspectives of institutional theory. The last section covers concluding comments and
implications for future research.
II. BACKGROUND AND OVERVIEW
Background
In recent years the major international public accounting firms have invested heavily in
ADA to transform their audit practice (Deloitte 2016; KPMG 2016; PwC 2017; EY 2017b). For
example, EY reports investment in new technologies in the total amount of $1billion (EY 2019).
Deloitte CEO Cathy Engelbert states: “Clearly all the audit firms are investing heavily in
innovation, transforming their audit business. … In the next five or six years I think that the
accounting and auditing profession will change more than it has in the last 30 [years]” (Cohn
2016). Similarly, Leonard Combs, PwC U.S. Chief Auditor and Leader of Auditing Services,
Methodology & Tools reports: “Data analytics is changing both the way we conduct our audits
and what those audits deliver. It allows us to extract and analyze ever-larger data sets. Further
use of data analytics will allow us to deliver effective audits more efficiently” (PwC 2017).
Additionally, there is a growing body of research that recognizes that ADA are likely to
significantly transform the conduct of the audit (e.g., Cao, Chychyla, and Stewart 2015;
Downloaded
from
by
Aasmund
Eilifsen
on
15
June
2020

7
Schneider, Dai, Janvrin, Ajayi, and Raschke 2015; Austin, et al. 2019; Barr-Pulliam, Brown-
Liburd, and Sanderson 2019; Salijeni et al. 2019). For example, Barr-Pulliam et al. (2019, 11-12)
note: “The use of advanced testing methods such as ADAs can occur at any stage of the audit
and can significantly transform the process of auditing financial statements, resulting in
enhanced audit effectiveness and audit efficiency—both elements and signals of audit quality.”
Likewise, Austin et al. (2019, abstract) state: “Data analytics is transforming our global markets.
Auditors, CFOs, standard-setters, and academics recognize this phenomenon and generally agree
that data analytics will fundamentally change financial reporting and auditing processes.” In the
same vein, Salijeni et al. (2019, 115) conclude that although “there is a danger of disappointment
if Big Data Analytics developments are seen as the golden ticket which is going to solve audit
problems and lead to a future in which audit services are universally valued. Equally, however,
these developments may provide real tools to reconfigure, refocus and potentially reposition
(within the firms' business models) contemporary audit practice.”
As a response to the firms’ emerging use of ADA, professional accounting bodies have
established task forces, issued position papers and alerts, and provided guidance for the growing
use of ADA on audits (e.g., AICPA 2014, 2015, 2017; IAASB 2016; ICAEW 2016; CPA
Canada 2017; IBR 2018). For example, the International Auditing and Assurance Standards
Board (IAASB 2016, 7) states: “In an increasingly complex and high-volume data environment,
the use of technology and data analytics offers opportunities for the auditor to obtain a more
effective and robust understanding of the entity and its environment, enhancing the quality of the
auditor’s risk assessment and response.” Additionally, the Chartered Professional Accountants of
Canada (CPA Canada 2017, 1) argue: “Auditors need to respond to ever-increasing and diverse
uses of information technology by audited entities. More use by auditors of technology-enabled
Downloaded
from
by
Aasmund
Eilifsen
on
15
June
2020

8
data analytics is an important aspect of that response.”
Supervisory bodies are also increasingly directing attention to the emerging use of ADA
by the firms (e.g., FRC 2017; PCAOB 2018). For example, the U.S. Public Company
Accounting Oversight Board (PCAOB) recognizes that auditors are rapidly expanding their use
of technology to plan and perform audits and one of its main goals in its “Strategic Plan 2018-
2020” is: “Anticipate and respond to the changing environment, including emerging technologies
and related risks and opportunities” (PCAOB 2018, 9). Similar to this paper’s focus, the U.K.
Financial Reporting Council (FRC 2017) undertook a review of the use of ADA on nineteen
2015 audits for the six largest U.K. audit firms. Overall, they found that firms relied on basic
ADA tools and found little or no use of advanced ADA tools.
Several researchers observe that little is known about the extent of ADA usage in audit
practice and how ADA affect the conduct of audits (e.g., Gepp et al. 2018; Austin et al. 2019;
Salijeni et al. 2019). Available research is mainly based on interview data. For example, Salijeni
et al. (2019) obtained evidence through semi-structured interviews conducted with regulators and
partners with significant experience in developing, implementing, or assessing the impact of
ADA in auditing. Their evidence indicates that ADA are utilized in areas of risk assessment,
journal testing, and full population testing. Similarly, Austin et al. (2019) interviewed managers,
external auditors, and standard-setters and found that companies and their auditors have
introduced data analytics to the benefit of financial reporting and audits, but lack of regulatory
guidance and implementation costs present considerable hindrances to implementation. Walker
and Brown-Liburd (2019) interviewed twenty-seven experienced internal and external auditors
and found that ADA usage is low given the arguments that ADA have the potential to increase
audit quality. Finally, CPA Canada (2017) surveyed the use of ADA from 394 auditors in the
Downloaded
from
by
Aasmund
Eilifsen
on
15
June
2020

9
practice (49 percent audit partners) covering large and small/medium-sized audit firms in
Canada. Although the use of advanced ADA was not common (20 percent of the respondents
have used advanced ADA), larger firms used more ADA than smaller firms, and some types of
ADA were used in all major phases of the audit, with the highest frequency of use in the
planning/risk assessment and in the substantive procedures phase (63 percent for large firms).
Large firms’ primarily used ADA as a corroborative source of audit evidence rather than as a
primary source. Overall, the CPA Canada study concluded that the audit profession is in the
early stages of mainstream adoption of ADA.
To summarize, audit firms, research scholars, accounting bodies, and regulators agree
that ADA likely will significantly transform the conduct of the audit. Research findings based on
interviews with managers, external and internal auditors, and standard-setters support this view
but also give indications that the expectations may not yet be fulfilled. However, little is known
about the stage of the ADA transformation in the field.8
Given the dearth of research on the
actual use of ADA in audit practice, we investigate its usage by gathering a rich data set from a
sample of 216 audit partners and managers and 109 audit engagements. Our work examines
factors related to the auditors’ perceptions of the use of ADA both within their firm and on the
sampled engagements, and ADA usage at the major phases of the audit. The research extends
prior studies by providing the first large sample study of the insights of partners and managers on
the use of ADA and extensive information on the actual use of ADA in the field.
Overview
The objective of the current research is to explore auditor perceptions of ADA and its
8
The U.K. Financial Reporting Council (FRC 2017) reviewed the use of ADA using a small sample (19
engagements) of the 2015 audits from the six largest U.K. audit firms. Sixteen of these audits arose from their
normal inspection process while a further three audits were nominated by audit firms to demonstrate a particular
ADA tool in use.
Downloaded
from
by
Aasmund
Eilifsen
on
15
June
2020

10
usage in current audit practice. Research goals and data availability are important determinants
for a preferred research method (Bloomfield, Nelson, and Soltes 2016). We use qualitative
research methods and field data to generate a grounded understanding of the use of ADA in the
field (Power and Gendron 2015). We felt that using both qualitative methods and field data were
necessary given the limited information about (1) auditors’ perceptions about ADA usage and (2)
the use of ADA in practice. Our work responds to calls for research to examine these issues (e.g.,
Gepp, et al. 2018; Salijeni et al. 2019).
Based upon our review of the academic and professional literature, we decided to take a
two-stage approach to meet the study’s objectives. In the first stage, we interviewed the heads of
professional practice for the five largest international public accounting firms in Norway to get
an understanding of the status of ADA within each firm. This stage of the study is described in
Section III. In the second stage, we obtained responses from the engagement partner and/or
manager to a detailed questionnaire about their perceptions about the acceptance and application
of ADA, including factors impeding the use of ADA and the use of ADA on the sampled
engagements. This stage of the research is described on Section IV.
III. INTERVIEWS WITH HEADS OF PROFESSIONAL PRACTICE
Participants
Our contact person within each firm indicated that the national head of professional
practice for the firms was the appropriate individual for the study, and each firm provided access
to those individuals. These individuals have the responsibility for the appropriate use of the
firm’s audit methodology, including implementing and monitoring ADA usage within the firm.
The Heads also work closely with groups within the firm’s global organization responsible for
the development of ADA tools. The five Heads were partners (3 males and 2 females) with 15 to
Downloaded
from
by
Aasmund
Eilifsen
on
15
June
2020

11
34 years of audit experience. They had been with their firms for more than ten years and had
been in their current position between three and ten years.
Method
The aim of this stage of the study was to develop first-hand knowledge about how the
major international firms are adopting ADA within their practices. Given the exploratory nature
of our research, we decided to do semi-structured interviews based on a script allowing for
answering broad and open-ended questions with the opportunity to follow interesting paths as
they develop during the interviews (Lillis 1999; Kenno, McCracken, and Salterio 2016). The
interview script was developed based on prior research and professional literature, discussions
with practitioners, and the considerable practice experience of one of the researchers as a prior
national head of audit and assurance for a Big 4 firm in Norway. A copy of the interview script is
provided in Appendix A. The interviews lasted from 48 to 92 minutes, with an average of 63
minutes. The interviews were audiotaped with permission of the interviewees.
Results
The interviews with the Heads indicated that major decisions on the extent of use of
ADA on a specific engagement are made by the engagement partner and/or manager.9
Thus,
those appropriate individuals served as participants for the second stage of our research.
The Heads confirmed what their firms’ press statements and webpages expressed that the
use of ADA was high on their agendas and that there is a general global firm push for ADA to be
used on audit engagements. However, the Heads also expressed that the use of ADA in the firms’
audits and its implementation into the audit methodologies are at an early stage and that we
should expect to observe varied use of ADA on the sampled engagements. None of the five audit
9
This was confirmed with the partners and managers who participated in the second stage.
Downloaded
from
by
Aasmund
Eilifsen
on
15
June
2020

12
firms had introduced mandatory use or mandatory pilot testing of advanced ADA tools.
The Heads indicated that they had not, on a general basis, approved ADA tools to be used
to provide the only form of audit evidence. The engagement partner and the audit team decide
whether the results from an ADA are sufficient appropriate audit evidence depending on the
circumstances. However, the Heads indicated that they frequently discussed with audit teams the
usage of and acceptance of ADA generated audit evidence.
The Head of Firm 110
said that the firm’s strategy for implementing ADA was extensive
training of the audit teams on the use of ADA tools where the teams locally would perform the
ADA. In contrast, the Head of Firm 2 reported that the firm had set up a specialist team, separate
from the audit teams, that collected, cleaned, and structured client data and provided a set of
standardized ADA analyses for the audit teams. Firm 3 had a similar arrangement as Firm 2 for
some large engagements. The Head of Firm 4 explicitly indicated that the firm was waiting
before actively pushing for implementation of new ADA tools in Norway because they felt it
was too costly to pilot new tools on engagements. They preferred to wait until other firm
countries had tested the new tools, found them workable, and could provide guidance on how to
efficiently use them on engagements. Further, the Head of Firm 4 did not experience strong
pressure from clients to use ADA. The Head of Firm 5 admitted that they had not come far in
ADA implementation, but were working hard on developing national tools that could extract data
from external databases and make automated, standardized analysis.
All the Heads reported that there was significant uncertainty about how the supervisory
authorities responsible for inspecting audit engagements (i.e., The Financial Supervisory
Authority of Norway (FSA) and PCAOB) would evaluate and accept ADA generated audit
10
In reporting the results, we have labeled the Firms 1 through 5 to maintain anonymity.
Downloaded
from
by
Aasmund
Eilifsen
on
15
June
2020

13
evidence.11
The FSA has been rather passive and silent on the issue. The Heads understanding is
that the FSA has not yet taken any position on the use of ADA on audits, including not signaling
or providing any guidance of appropriate ADA usage. The FSA had informed several of the audit
firms that their strategy is to inspect completed audit engagements and then form their own
opinion on the appropriateness of the ADA usage. This made several of the Heads skeptical to
use evidence generated from ADA as the only form of evidence to support assertions. For audit
engagements subject to PCAOB supervisory authority, inspection risk has led all the firms to
inform the audit teams not to use evidence generated from ADA as the only form of audit
evidence.
IV. AUDITOR PRECEPTIONS OF ADA AND USAGE ON AUDIT ENGAGEMENTS
Methods
The methods used in stage 2 involved completion of a detailed questionnaire addressed to
the engagement partners and managers followed by an open (semi-structured) discussion of the
issues. We first developed a preliminary questionnaire based on the responses from the
interviews with the Heads and our understanding of the literature, including prior studies of
ADA use12
and the more general studies about the acceptance and adoption of emerging
technologies. We then discussed the preliminary questionnaire with the Heads and made any
necessary adjustments. Pilot testing resulted in some minor adjustments to the questionnaire.13
11
The Financial Supervisory Authority of Norway (FSA) is the governmental agency in charge of inspections and
enforcement for statutory audits in Norway but FSA is not a standard setter. The EU’s accounting and auditing
regulations apply in Norway, and all Norwegian audits should comply with the IAASB International Standards on
Auditing (ISAs). The PCAOB and FSA have a formal cooperative arrangement where they carry out inspections
jointly on accounting firms located in Norway that audit, or participate in audits, of companies whose securities
trade in U.S. markets.
12
We thank Clark Hampton for providing us with the survey instrument for investigating the use of ADA within the
Canadian auditing profession (CPA Canada 2017).
13
Five audit managers with an average of seven years of audit experience and three master students with audit
experience took part in the pilot testing.
Downloaded
from
by
Aasmund
Eilifsen
on
15
June
2020

14
The questionnaire consisted of three parts and was administered using Qualtrics.14
The first part
of the questionnaire included instructions and questions to be answered separately by the
engagement partner and manager. The instructions outlined the purpose of the study, explained
the structure of the questionnaire, referred to the definition of ADA (AICPA 2014), and specified
the scope of ADA for the purpose of the survey.15
The first set of questions related to
participants’ demographic data. Next, the participants were asked to rate on a 7-point Likert scale
(1 - Strongly disagree, 4 - Neutral, 7 - Strongly agree) their competence in information
technology (IT) systems and ADA tools overall. This was followed by a set of statements about
(1) perceived competency (knowledge of and training in use of firm available ADA) and (2)
perceived usefulness of ADA. The participants were asked whether they disagreed or agreed
with the statements on a 7-point Likert scale (1 - Strongly disagree, 4 - Neutral, 7 - Strongly
agree) for (1) audits generally within their firm and (2) the specific audit engagement selected.16
Participants were also asked about their perceived influence on the decisions of ADA use for the
selected engagement and whether the ADA use differed relatively to the ADA use for their other
audit engagements.
The second and third parts of the questionnaire related specifically to the selected
engagement and were answered by the engagement partner and/or manager. Note that these two
parts of the questionnaire were completed by the participants who had direct access to the audit
14
The study was approved by the Norwegian Centre for Research Data (NSD). The instructions to the questionnaire
made it clear that the survey was voluntary and that participating individuals, the specific engagement(s), and firm
will remain anonymous in reporting the results of the study. A copy of the questionnaire is available from the
authors.
15
We provided examples of where ADA could be used. These included ratio analysis, trend analysis, regression
analysis, general ledger/account reconciliation and analysis, journal entry analysis, segregation of duties analysis,
three-way match procedure, cluster analysis, and data mining. We excluded manual analysis, simple Excel
calculations, auditor’s tests of the client’s systems or tools, tools which only automate a process at the audit firm,
and the process of extracting, securing, preparing, and storing client or external data at the audit firm.
16
The statements were randomized to control for order effects.
Downloaded
from
by
Aasmund
Eilifsen
on
15
June
2020

15
workpapers. The second part collected general information about the client (type of entity,
industry, size, audit hours used, etc.), details of the client’s accounting information systems, and
if and how data from the client’s systems was extracted and used in the audit, including any
application of ADA in this process. This part of the questionnaire also included questions related
to the use of ADA on the engagement selected on the client acceptance and continuance
decisions, and fraud risk assessments.
The third part of the questionnaire covered the use of ADA in each phase of the audit,
from planning to completion for the selected engagements. For each phase where ADA use was
reported, a number of follow-up questions were posed. These questions included details of the
types of ADA used, hours used for the ADA, whether the ADA relied on client or other external
sources for data, whether the usage of the ADA implied a change in audit procedures compared
to prior year, whether the use of the ADA was mandatory or not, whether the ADA used were
firm global tools or developed nationally, whether the applied ADA were used in other audit
engagements, which team members that primarily performed the ADA, and whether the use of
the ADA was considered to affect the efficiency and/or effectiveness of the audit.
After completing the questionnaire, participants present at the firm sessions (see below)
were asked some semi-structured follow-up questions. This offered the participants an
opportunity to comment on the survey and otherwise express any views related to the study. One
or two of the researchers were present and the discussions were audiotaped.
Sample Engagements
The firms agreed to provide a sample of audit engagements with participation of the
engagement partner and manager. Each firm was asked to select six engagements within each of
Downloaded
from
by
Aasmund
Eilifsen
on
15
June
2020

16
the following four categories:17
• ADA: Engagements where the audit firm expected high ADA use.18
• Listed: Largest engagement entity (a parent company or a subsidiary) based on
revenues of a listed company (group).
• Large_NL (non-listed): Largest engagement entity (a parent company or a subsidiary)
based on revenues of a non-listed (NL) group with group revenues above $100
million.
• Small_NL (non-listed): Engagement entity of a non-listed (NL) group with group
revenues below $10 million, or a non-listed (NL) entity not part of a group with
revenues below $10 million.
With the exception of the high ADA use engagements, which were identified by the
Heads, engagements were selected randomly by each firm from an alphabetized list of audit
clients. Of the 120 engagements selected, our final sample includes 109 engagements.19
Ninety-
six of the 109 questionnaires were completed in sessions held for each firm (twelve engagement
teams participated by Skype or similar communication platforms at the firm regional office
location), and 13 questionnaires were completed outside of the firm sessions. Table 1 provides
the distribution of the number of engagements across categories of engagements.
[Insert Table 1 here]
Table 2 (Panel A) provides descriptive information about the 109 sample engagements
representing a large variety of industries with retail/wholesale (19); transportation, shipping,
distribution (13); and manufacturing (10) as the three most represented industries. For industries
with more than five selected engagements in total (N), engagements with expected high ADA
17
The audit firms were clear that the selected unit for investigation of ADA use should be individual entities rather
than a group. Although the group engagement team establishes the overall group audit strategy that may involve
decisions about the use of ADA for entities in the group, ADA use for entities within a group may differ due to
entity differences such as for example entities’ ERP systems. The Listed and Large_NL engagements entities are the
largest (by revenues) Norwegian entity (a parent company or a subsidiary) within a group. The ADA and Small_NL
engagement entities include the largest (by revenues) Norwegian entity within a group and “stand-alone” entities not
part of a group (see Table 2, Panel B).
18
Of the 28 ADA engagements, 8 were listed (4 of 8 were newly tendered audits) and 20 were non-listed. Of the
non-listed companies, 16 were “Large” and 4 were “Small.”
19
Three engagements did not participate, and eight engagements were excluded due to incomplete questionnaire
responses.
Downloaded
from
by
Aasmund
Eilifsen
on
15
June
2020

17
use have the highest relative representation in financial services (50.0 percent), software (42.9
percent), retail/wholesale (42.1 percent), and manufacturing (40.0 percent).
Panel B presents information on the size of the engagements based on revenues in 2017,
and the entities’ group affiliation. Large_NL (non-listed) engagements have the highest average
revenues followed by ADA engagements. Listed engagements are more evenly distributed across
revenue brackets while Small_NL (non-listed) engagements, with one exception,20
have less than
$10 million in revenues. Group affiliations are: 14 entities not part of a group (ADA or
Small_NL engagements), 37 parent companies, 32 subsidiaries of a Norwegian parent company,
and 26 subsidiaries of a non-Norwegian parent company.
Panel C presents the average audit hours for the 109 engagements (mean=891 hours).
Average audit hours used for Listed and Large_NL engagements (mean=936 hours, mean=958
hours, respectively) are slightly above the sample average. The average audit hours used for
Small_NL engagements (mean=95 hours) were approximately one tenth of the sample average.
In contrast, ADA engagements average audit hours (mean=1,575 hours) was 77 percent more
than the sample mean. Although the majority of the engagements with expected high ADA use
were clients with relatively high revenues, a much lower number of audit hours were reported for
Large_NL engagements with the highest average revenues.
Participants
Our sample includes 216 participants. For 92 of the 109 engagements both the partner
and manager completed all parts of the questionnaire; the first part of the questionnaire
20
The selection of engagements was in some cases based on the 2016 revenues since the 2017 revenues number had
not been prepared at the date of engagement selection. One engagement in the Small_NL category reported more
than $10 million in revenues for 2017.
Downloaded
from
by
Aasmund
Eilifsen
on
15
June
2020

18
individually and the second and third parts together. For 15 of the engagements, the first part of
the questionnaire was completed individually by the partner and the manager, while the manager
completed the second and third part of the questionnaire that was later reviewed by the partner.
Two engagements were completed only by the manager.
Table 3 presents the demographic data of the participants. 177 (82 percent) males and 39
(18 percent) females participated. Nighty-two percent of the participants were 30 years or older.
Most participants (92 percent) held one or more master’s degrees.21
Ninety percent of the
participants had more than five years of audit experience. The large majority of our participants
were leaders of the audit engagement team. Eighty-six percent of the participants had the rank of
managers or higher, and the remaining 14 percent had the rank of seniors or lower. Seventy-eight
percent of the participants’ overall audit time was allocated to listed companies (34 percent) and
large (23 percent) or medium sized non-listed entities (21 percent).
Auditor Perceptions of ADA
We gathered a substantial amount of information about the participants’ perceptions of
ADA. The first part of the questionnaire asked the partner and manager to separately respond (1)
to a number of questions about their competence in IT systems and ADA tools overall and (2) to
a series of statements related to perceived competency (knowledge and training) and perceived
usefulness of ADA, both for audits in general within their firm and the specific audit engagement
selected. In addition, we asked about the participants’ perceived influence on the decisions to use
ADA on the selected engagement.
21
State authorization is the highest level of auditor certification in Norway and requires a master’s degree in
accounting and auditing. It is not uncommon that a state authorized public accountant holds a second master’s
degree in business administration or other discipline.
Downloaded
from
by
Aasmund
Eilifsen
on
15
June
2020

19
Table 4 presents the auditors’ responses on a 7 point Likert scale (1 - Very Low, 7 - Very
High)22
to four questions related to their perceived competence with IT systems and ADA tools:
(1) IT systems and Enterprise Risk Planning (ERP) systems, (2) Data programming, data
handling, and data storage, (3) Interpreting the results form data analytics in general, and (4)
ADA tools which are available to you within the firm. We tested the responses by comparing the
mean ratings against the midpoint (4) of the scale and by comparing partners with managers.
The overall perceived competence (4.09) with ERP/IT systems was not significantly
different than the midpoint while the partners’ perceived competence was significantly lower and
the managers’ significantly higher than the midpoint (p<0.05). The partners’ perceived
competence was significantly lower than the managers (3.82 versus 4.27; p<0.05). The overall
perceived competence with data programming, data handling, and data storage was significantly
lower than the midpoint overall (3.43, p<0.05) and for the partners and managers individually
(3.27 and 3.60; p<0.05). The partners’ and managers’ perceptions did not differ significantly
from each other. In contrast, the overall (4.80), and partners’ (4.70) and managers’ (4.89)
perceptions of competence with interpreting the results from data analytics were significantly
higher than the midpoint (p<0.05) while the partners’ and mangers’ perceptions did not
significantly differ. For perceived competence with ADA tools available within the firm, the
overall and managers’ perceptions were significantly above the midpoint (p<0.05) while the
partners’ perceptions were not. The managers’ perceptions were significantly higher than those
of the partners (p<0.05).
22
On the questionnaire, the scale used for these questions was (1 – Very High to 7 - Very Low). We reversed it to
make the presentation of the results easier to understand.
Downloaded
from
by
Aasmund
Eilifsen
on
15
June
2020

20
Table 5 presents the participants’ ratings on a 7 point Likert scale (1- Strongly disagree, 4
- Neutral, 7 - Strongly agree) for a series of statements related to their perceived competency
(Panel A), the usefulness of ADA for audits in general within their firm (Panel B), and for the
selected engagements (Panel C).23
Table 5, Panel A, shows that the partners and managers
perceive that they have sufficient knowledge to use the ADA tools available to them within their
firm (4.42; p<0.05). For the other three statements on competence their ratings are basically
neutral (not significantly different from the midpoint).
An overall examination of the participants’ responses to the statements about the
usefulness of ADA (Table 5, Panel B) indicates positive perceptions of the usefulness of ADA
for firm audits in general. For example, the participants tend to disagree that ADA tools are
complex and difficult to use and are neutral towards the statement that the ADA tools are very
time-consuming to use on engagements. They generally agree more than they disagree that their
firms have incentives and/or pressure to use ADA instead of traditional audit tools, that current
auditing standards accept the use of audit evidence from ADA, their firms’ methodology accepts
ADA evidence, supervisory bodies accept ADA as audit evidence, ADA are useful for risk
identification and substantive testing, and they are not hesitant to use ADA. In addition, they
agree that their firms have adequate ADA tools, that these ADA tools improve the quality of
audits, and that ADA tools are efficient. In line with the partners’ and managers’ confidence in
their own knowledge of firm ADA tools and their usage, they disagree that their audit teams lack
technical competence to use ADA tools and agree that their firms have sufficient specialist
23
We do not tabulate the means for the statements for the individual firms and partners and managers but report
significant differences (p<0.05) in the text.
Downloaded
from
by
Aasmund
Eilifsen
on
15
June
2020

21
support. Finally, participants agree that clients have a positive attitude towards the use of ADA
but are neutral towards the statement that it is easy to collect sufficient data for ADA purposes.
Overall, we observe a higher perceived competence and usefulness of ADA for participants of
Firm 1 and Firm 2 compared to the other three firms. This may relate to the fact that these firms
having a more “active” approach to the implementation of ADA. Firm 1 trains audit team
members extensively in ADA use while Firm 2 has set up a general function of specialists that
prepare data and provide a set of standardized ADA analyses for the audit teams.
Table 5, Panel C presents the responses to a series of statements, similar to those in the
last ten rows in Panel B related to the perceived usefulness of ADA for the selected
engagements. Overall, participants perceived the usefulness of ADA for the selected
engagements positively but to a lesser degree compared to firm audits in general. Also, for the
selected engagements Firm 1 and Firm 2 scored highest on perceived usefulness of ADA.
The partners and managers were also asked the following question: “How influential, in
your own opinion, are your decisions of the actual usage of ADA on the specific engagement
selected for this survey? [] Highly influential [] Very influential [] Somewhat influential []
Limited influence; other team members mostly decide [] Limited influence.” Forty-five percent
of the participants answered highly influential and thirty-eight percent answered very influential.
Thus, it is clear that the engagement team leaders (partner and manager) are very influential in
the decision to use ADA on the selected engagements.
ADA Use on Audit Engagements
Client IT Systems and ADA Data Collection
Table 6 presents information about the entities’ ERP/IT systems and a series of questions
related to the collection of data for ADA use on the sampled engagements. We present the results
Downloaded
from
by
Aasmund
Eilifsen
on
15
June
2020

22
by the type of engagement and report untabulated findings by firm when relevant. All
engagements (109) report using some information from the entity’s ERP/IT systems in the audit.
Twenty-eight engagements did not use ADA in any phase of the audit, thus leaving eighty-one
engagements which used ADA in the audit. The twenty-eight engagements not using ADA
include 10 Listed engagements, 8 Large_NL (non-listed) engagements, and 10 Small_NL (non-
listed) engagements.
Table 6 Panel A shows that for Small_NL and to some degree Large_NL engagements
the general ledger is far less integrated with other accounting systems compared to ADA and
Listed engagements. The majority of the clients have highly integrated (41.3 percent) or partially
integrated (37.6 percent) ERP/IT systems. Clients selected as ADA engagements have the most
integrated ERP/IT systems followed by Listed and Large_NL engagements. On average ADA
were used eleven times for engagements with highly integrated ERP/IT systems and six times for
other engagements (untabulated). This suggests that a client needs to have a high degree of
ERP/IT system integration to provide opportunities to use ADA. Outsourcing of the accounting
system is most common for Small_NL engagements and in these cases the accounting system is
typically highly integrated with other systems.
Panel B reports the accounts or records where data were collected for ADA use for the
eighty-one engagements where ADA were used. All but one of the eighty-one engagements
collected data from the general ledger. Twenty-six and twenty-five engagements collected data
from accounts payables and account receivables, respectively, while relatively few engagements
collected data from other accounts. One hundred ninety total accounts or records were accessed
for the eighty-one engagements using ADA.
Downloaded
from
by
Aasmund
Eilifsen
on
15
June
2020

23
Panel C shows who collected the data for ADA use. The client collected the data based
on instructions from the auditor on 35 engagements (43.2 percent). Firm IT specialists collected
the data on 26 engagements (32.1 percent), audit team members for 17 engagements (21.0
percent), and others (e.g., external service providers) for 3 engagements (3.7 percent).
Panel D reports who prepared that data for ADA use.24
The vast majority of the data
preparation (95.1 percent) was completed by firm IT specialists (58.0 percent) and audit team
members (37.1 percent). In contrast to collection of data for ADA use, only for one engagement
was the client involved in the data preparation process. Panel E shows that audit team members
conducted most the ADA work (69.1 percent) complemented by IT specialists (29.7 percent).
Finally, we observe that for Firm 1 (i.e., the firm that provided extensive training of ADA for
audit team members) 95.8 percent of the engagements reported that the audit team members did
the main part of the ADA work. Firm 2 (i.e., the firm that used specialist team to assist the
engagement team) reported that the specialist team did the majority of the ADA work on the
engagements (62.5 percent).
ADA Usage in the Phases of the Audit
Table 7 reports on the use of ADA in the major phases of the audit engagements (i.e.,
client acceptance and continuance decisions, fraud risk assessment, internal controls, substantive
testing, and final review) and the audit steps within each phase. We present the results by the
category of engagement and by firm.
Less than a quarter of engagements (25 of 109 engagements) use ADA in client
24
Untabulated results show that the number of hours used to collect client data was not very high. About sixteen
percent of the engagements that collected data from the client used more than 20 hours. Eighty-one percent of the
engagements used less than 20 hours to clean, structure, and store the collected data.
Downloaded
from
by
Aasmund
Eilifsen
on
15
June
2020

24
acceptance and continuance decisions. Most common ADA tools in use were firm global
template tools and database management systems (i.e., system software for creating and
managing databases). Less ADA are used for listed companies than for the other categories. This
is consistent with reluctance to use ADA due to higher inspection risk exposure for listed
companies. The use of ADA in client acceptance and continuance decisions varies considerably
between firms. Firm 4 did not use ADA for such purposes while Firm 3 used ADA on 44.0
percent of their engagements. Only 19 of the 109 engagements use ADA for fraud risk
assessments with most frequent use by ADA engagements. Firm global template tools and
database management systems are the most commonly used ADA tool for fraud risk
assessments. Variation exists between firms in the use of ADA for fraud risk assessments, from
10.5 percent of engagements for Firm 2 and 4 to 36.8 percent for Firm 1. The low overall use is
surprising given that ADA have been promoted as particularly valuable in fraud risk assessments
and prediction (e.g., AICPA 2017; Perols, Bowen, Zimmermann, Samba 2017; Austin et al.
2019).
The planning phase includes seven audit steps. These are ordered by frequency of ADA
use: overall assessment of client's operations and performance (33 engagements), identifying and
assessing key risks (27 engagements), mapping of different business processes (26
engagements), determining and allocating materiality (18 engagements), overall assessment of
industry risks and understanding the business (17 engagements), assessing going concern issues
(11 engagements), and other planning (4 engagements). Although ADA are used to some extent
for understanding the client and in risk assessments, the numbers are low as these are areas
where the Heads expected ADA to be used. The variation between firms is considerable. Firms 1
and 2 were the most frequent users of ADA in audit planning phase.
Downloaded
from
by
Aasmund
Eilifsen
on
15
June
2020

25
Thirty-seven engagements used ADA to evaluate and test internal controls using
extracted client data, with the most frequent use by ADA engagements (17 of the 37
engagements). ADA usage related to internal controls was fairly evenly distributed across firms.
The substantive testing phase was divided into nineteen audit steps. The steps with the
highest usage of ADA include: journal entry testing (74 engagements); calculating sample-size
for testing (55 engagements); selection of random samples from populations (51 engagements);
summarizing ledgers by computer audit tools (41 engagements); simple judgmental methods
(e.g., deviations from average, larger/smaller, negative) (40 engagements); visual (non-
statistical) identification of deviations, outliers, groups or similar (37 engagements); two or
three-way matching of specific amounts (36 engagements); reconciliations (e.g. matching total
numbers between different ledgers or similar) (33 engagements); recalculating, re-performing,
testing of formula by computer audit tools (27 engagements); and making statistical calculations
from the data for predictive analysis (21 engagements). The findings indicate that ADA usage is
common for some steps in the substantive testing phase. Some of these analytics have been used
in audits for decades while others reflect more recent ADA. Consistent with the firm approaches
to implement ADA, Firm 1 and 2 were the most frequent users of ADA in the substantive testing
phase.
Eight audit steps were included in the final review and conclusion phase. ADA were used
most in reconciliation and control between final accounts and underlying ledger (26
engagements) and for analytical checks and reasonableness checks of results (25 engagements)
with again Firm 1 and 2 as the most frequent users.
On average, ADA were used seven times in each engagement with highest average on
ADA engagements (9.6 times). The average number of times that ADA was used on
Downloaded
from
by
Aasmund
Eilifsen
on
15
June
2020

26
engagements for Firm 1 to Firm 5 was 11.7, 7.7, 5.1, 4.6, and 5.5, respectively. This mirrors the
firms’ strategy for implementing ADA.
Table 8 shows which type of ADA tool and the number of times they were used by
engagement category and firm. Excel with basic functionality is used 424 times as an ADA tool
(representing 34.4 percent of the total 1,232). More advanced Excel (including Power-BI, Solver,
Miner, and Macros) is used 319 times (25.9 percent of the total). More sophisticated software
with capacity to handle Big Data is used less. Statistical packages (e.g., ‘R’, Stata, SPSS);
programming languages (e.g., Python and Pearl), and visualization programs (e.g., Tableau and
Spotfire) are used 2.7 percent, 2.2 percent, and 10.6 percent of the total times, respectively.
“Other” ADA tools (8.1 percent of the total) include firm specific tools — PACE (EY), Halo
(PwC), KAAP (KPMG), and Spotlight (Deloitte). According to the firm Heads these tools
include some of the tools described above. Business Intelligence (BI) Analytics Management
Software (e.g., IBM, Oracle, SAP, SAS) and database management systems (e.g., Access,
MySQL, SQLServer, Oracle) were used 6.9 percent and 9.2 percent of the total times,
respectively. ADA tools were used most on ADA engagements (441 times of 1,232; 35.8
percent) and these engagements have the highest frequency of use of advanced Excel, database
management systems, visualization programs, and “Other”’ tools. As expected, Firm 1 uses
ADA tools most frequently (39.7 percent of the total number of times), followed by Firm 2 (19.9
percent), Firm 3 (15.3 percent), Firm 5 (14.0 percent), and Firm 4 (11.0 percent).
Untabulated results indicate that statistical packages are mostly used for statistical
sampling or journal entry testing and that visualization tools are, in many cases, used for journal
entry testing, non-statistical visualization, or simple visualization of deviations.
Downloaded
from
by
Aasmund
Eilifsen
on
15
June
2020

27
During the semi-structured discussions that followed the completion of the questionnaire,
participants drew our attention to ADA use and the tender process. One participant made the
following comment: “When we have written about data analytics all over the place in the audit
tender and promised to use it in the presentations - well, then you just have to use it when you
have won the audit.” Based on the comments about the tender process, we contacted each firm
and asked how many of the 109 engagements were a new client in the years 2016 or 2017.
Nighty-five were continuing engagements and fourteen were new tendered engagements in 2016
and 2017. Table 9 shows engagement perceptions of the usefulness of ADA (cf. Table 5, Panel
C) and the number of times ADA were used in the auditing process (cf. Table 7, last row) for
continuing and new engagements.
ADA usefulness is systematically perceived higher for all statements for new compared
to continuing engagements. On average, ADA were used fourteen times for new engagements
and six times for continuing engagements. Based on the discussions in the firm sessions, the
reasons for the higher use of ADA on new engagements seem to be a combination of
expectations from the clients, commitments in the tender process to the use ADA by the winning
audit firm, and the costs of changing audit approach and procedures on continuing engagements.
Viewing ADA Use through an Institutional Theory Perspective
To better understand the low use of ADA on audit engagements and develop insight into
the underlying causes of the problems facing the auditor in the decision to apply ADA and how
these can be mitigated, we relate our findings and analysis to perspectives from institutional
theory.25
The term “institutional theory” covers a broad body of literature. Wotton and Hoffman
25
The accounting information system literature has tested several theories about the acceptance and adoption of
Downloaded
from
by
Aasmund
Eilifsen
on
15
June
2020

28
(2017, 55) state: “What these literatures have in common, however, is an underlying skepticism
towards atomistic accounts of social processes, relying instead on a conviction that institutional
arrangements and social processes matter in the formulation of organizational action (DiMaggio
and Powell 1991).” At its core, the institutional theory literature looks at the source of action as
existing exogenous to the actors (Wooten and Hoffman 2017).
Legitimacy is a fundamental concept of organizational institutionalism and refers to how
organizations seek legitimacy by its audience (constituencies) of its activities.26
Institutional
theory-based auditing research has focused on the conception of legitimacy and legitimation
processes of audit practices (Power 1997, 2003). This legitimacy-oriented perspective views the
interrelationship between audit practices and attempts to secure its legitimacy as central to
development and widespread acceptance of emerging practices (O’Dwyer, Owen, and Unerman
2011). Several studies have empirically investigated how audit practices seek to gain the
legitimacy by key constituencies (e.g., auditees, regulators, and supervisory inspection
authorities) necessary for them to prosper (e.g., Curtis and Turley 2007; Robson, Humphrey,
Khalifa, and Jones 2007; O’Dwyer et al. 2011; Griffith, Hammersley, and Kadous 2015; Salijeni
et al. 2019). Operating in a highly regulated and supervised industry and acting in the public
emerging technologies (e.g., Goodhue and Thompson 1995; Dowling 2009; Zhou, Lu, and Wang 2010; Bierstaker,
Janvrin, and Lowe 2014; Payne and Curtis 2017; Li, Dai, Gershberg, and Vasarhelyi 2018). Our study is of an
exploratory nature and these theories typically require a large and structured data set to test. In a recent study
examining how data analytics influence interactions among user groups (i.e., managers, auditors, and regulators),
Austin et al. (2019) interpret their interview findings through socio-technical systems of innovation theory (S-T
theory), drawing on sociology and institutional theory as advanced by Geels (2004).
26
The literature contains several definitions of the concept of legitimacy. Based on a review of the literature
Deephouse, Bundy, Tost, and Suchman (2017, 32) offer the following definition of legitimacy: “Organizational
legitimacy is the perceived appropriateness of an organization to a social system in terms of rules, values, norms,
and definitions.” Suchman (1995) identifies three primary forms of legitimacy that an organization may use to gain
legitimacy by its audience (constituencies) of its activities. Pragmatic legitimacy relates to whether organizational
activity benefits the audience; moral legitimacy relates to whether organizational activity is normatively approved
by the audience; and cognitive legitimacy relates to comprehensibility (predictability and plausibility) of
organizational activity and whether of organizational activity is taken for granted (necessary or inevitable) by the
audience. Our focus is primarily on pragmatic legitimacy.
Downloaded
from
by
Aasmund
Eilifsen
on
15
June
2020

29
interest, audit practices need legitimacy. Salijeni et al. (2019) argue that the rise of ADA should
be understood as auditors’ attempts to promote technological solutions for the problem of
maintaining (or restoring) the legitimacy of the audit function. Walker and Brown-Liburd (2019)
propose a conceptual framework that comprehensively describes the emergence of the use of
ADA by audit professionals through the lens of institutional theory.27
They argue that in the
process of incorporating ADA into the audit, firms respond to pressure from the external
environment and attempt to gain legitimacy of ADA usage within their environment.28
Given
environmental pressure and firms’ legitimation process, the individual auditor acts and decides
whether it makes sense to use ADA in the audit. Walker and Brown-Liburd (2019) refer to
sense-making theory to investigate how practitioners decide and act (or do not act) to use of
ADA. In the following discussion of our results we draw on the view above that institutional
(environmental pressure), organizational (audit firms’ legitimacy process), and auditor specific
factors (sense-making) influence the actual ADA usage on audit engagements.
Accounting firms perceive pressures from their institutional environment that will likely
influence their decision around the adoption of ADA.29
It is clear from the firms’ public
announcements and their investments in ADA that technological advancements, the richer
sources of data available, and expectations of providing effective audits more efficiently in a data
analytic intensive business environment, represent significant technological pressures on the
27
Walker and Brown-Liburd (2019) relate their conceptual framework to interview responses from 27 experienced
internal and external auditors.
28
This reasoning is in line with Griffith et al. (2015, 836) who apply institutional theory to examine audit of
accounting estimates and state: “Institutional theory posits that organizations respond to the pressures and
constraints of their environments by developing and changing existing practices that establish or enhance the
organizations’ legitimacy within the environment.”
29
DiMaggio and Powell (1983) suggest three forms of institutional pressures for isomorphism; coercive (stemming
from political influence), mimetic (resulting from standard responses to uncertainty), and normative (associated with
professionalization). The perspective has also been used to explain how and why organizations change established
practices (e.g., Greenwood, Suddaby, and Hinings 2002). Our main focus is on coercive pressures exerted by key
constituencies to change (or not change) to ADA usage.
Downloaded
from
by
Aasmund
Eilifsen
on
15
June
2020

30
firms to adopt ADA. All the heads of professional practice of the five international public
accounting firms confirm that there is a global firm push for ADA to be used on audit
engagements. At the individual level, the audit team leaders (partners and managers) perceive
that the firms give incentives or exert pressure to use ADA instead of traditional audit tools.
Audit client pressure may also influence their decision around the adoption of ADA although all
heads of professional practice did not agree on such pressure. We observe that more ADA are
used for clients where the accounting system is fully integrated with other ERP/IT systems (e.g.
payroll/HR, inventory, production, fixed assets). Client integrated ERP/IT systems both force
and give the auditor an opportunity for cost-efficient use of ADA. Further, a higher frequency of
ADA use is observed for new audit engagements that have recently been tendered. This may
reflect that management and the audit committee exert pressure for use of ADA in the tendered
audits but also firm peer pressure to commit to use of ADA in the tendered audit. Overall, the
audit team leaders perceive the usefulness of ADA to be higher on new engagements. In addition
to client pressure for ADA usage and commitment to use ADA in tendered audits, cost-
efficiencies of ADA use on new versus continuing engagements may explain this attitude. We do
not observe that it is perceived that standard-setters or supervisory inspection authorities exert
pressure to apply ADA in the audits. Rather uncertainty of supervisory authorities’ position of
ADA usage in the audit evidence gathering process is perceived as preventing or limiting its use
because of potential detrimental consequences of inspections concluding with non-compliance
with regulations and standards.
The firms’ Heads express significant uncertainty about how the supervisory authorities
will evaluate and accept ADA generated audit evidence. They indicated that the supervisory
authorities are passive and silent on the use of ADA in the audit. The supervisory authorities in
Downloaded
from
by
Aasmund
Eilifsen
on
15
June
2020

31
Norway, FSA, have informed several of the audit firms that their strategy is to inspect completed
audit engagements and then form their own opinion on the appropriateness of the ADA usage.
For audit engagements subject to PCAOB supervisory authority, supervisory risk has led all the
firms to inform the audit teams not to use ADA as the only form of audit evidence. The partners
did not express strong views on whether the supervisory authorities and the current auditing
standards accept the use of audit evidence derived from ADA. We did not identify that the firms
had developed a legitimation strategy of ADA usage towards the supervisory authorities. Thus,
the primary arena for the accounting firms’ search of legitimacy of ADA usage seems currently
towards audit clients and particularly new clients. It is not clear from our data; however, whether
the client or the tendering audit firm is the most proactive in pressing for ADA usage during the
tender process. Further, no firm has introduced mandatory use of advanced ADA tools in their
audits and the firms differ in their strategies on how they implement the use of ADA in their
practices; from a “wait and see” position to more active firm involvement to facilitate ADA use.
The firms appear to act cautiously towards implementation of more advanced ADA. We
therefore conclude that legitimization of ADA usage is not yet of high priority for the surveyed
firms, except for promoting externally in press statements, speeches, publications, and on their
webpages that they are engaged in the process of transforming to ADA usage in the audit,30
and
at least to some extent in search of legitimacy of ADA usage towards audit clients, particularly
new clients.31
30
This concurs with Salijeni et al.’s (2019) conclusion the firms promote technological solutions for the problem of
maintaining (or restoring) the legitimacy of the audit function.
31
The firms’ “passive” legitimization strategy towards regulators and supervisory authorities may reflect that this is
considered an issue to be primarily handled at the international rather than the national level. For example, IAASB is
currently exploring issues arising from technology in conjunction with audit evidence gathering (IAASB 2019).
PCAOB has established a Standing Advisory Group (SAG) to discuss the use of data and technology in audits and
the potential implications for PCAOB auditing standards. Committee of European Audit Oversight Bodies
(CEAOB) has on its agenda: “Analysis of the impact of new audit tools and technologies on audit quality and
Downloaded
from
by
Aasmund
Eilifsen
on
15
June
2020

32
Given environmental pressure, the firm’s legitimization strategy, and other considerations
such as the audit team’s ADA skills, specialist resources available, adequate available firm ADA
tools, audit client characteristics (e.g., integrated ERP/IT systems or not), money and time
available within the engagement budget; the partner and managers decide whether to use ADA
on a specific engagement. The firms encourage but do not require, the use of ADA tools and the
decision of ADA usage is left to the leaders of the audit team. We are not able to substantiate that
the partners and managers find firm available ADA tools inadequate, available firm specialist
support (outside the core team) insufficient, or their ADA competence a hinder to the use of
ADA tools. The ability of the ADA tools to efficiently and effectively provide sufficient and
appropriate evidence is fundamental in the auditor decision to apply ADA in the audit (i.e., ADA
usage must make sense for the practitioner given regulations, firm requirements, and the above
mentioned considerations such as budget and client characteristics).32
The need to make sense by
those in the organization who will use the new technology intensifies in circumstances where
individuals face unexpected situations in their organizations, experience a high degree of
ambiguity, and there is no predetermined way to act (Weick, Sutcliffe, and Obstfeld 2005).
Uncertainty and ambiguity are both important attributes in the decision to adopt ADA (Walker
and Brown-Liburd 2019). The audit team leaders with intimate knowledge of regulations,
inspection risk, and firm policies, are likely to be aware of and consider the limitations of the
usage of ADA tools in the evidence gathering process and therefore act cautiously in adopting
auditor oversight as well as cyber security risks.” (CEAOB 2018, 9).
32
This argument is line with Curtis and Turley (2007, 440) that in their study of the implementation of business risk
audit at engagement level, point out that from the practitioner perspective he or her “may be concerned that the
process not only meets firm determined standards on methodology, but also actually provides a body of evidence
that the practitioner believes to be valid as a basis for signing the audit opinion and as a defensible record of the
audit process.”
Downloaded
from
by
Aasmund
Eilifsen
on
15
June
2020

33
extensive use of ADA.33
A partner made following statement: “You know, we all know we
should use data analytics in the audit. But when you are not quite sure how to do it, you know it
is costly to implement, and you are not quite sure what the supervisory authorities will say. Well,
then you wait a bit.”
Based on the arguments in the preceding and illustrated by the quote above, we suggest
that the limited use of ADA will persist until it is incorporated internally in the firms’
methodology and reward systems; explicitly supported and accepted by standard-setters and
supervisory bodies; and practitioners find that ADA use proves efficient and effective on the
engagement. Thus, it is likely that institutional changes directed to reduce perceived inspection
risk related to ADA usage and incorporation of ADA into the firm methodologies, as well as
convincing insight into the gains of ADA use in evidence gathering, are necessary to advance to
an extensive use of ADA in the audit process.
V. CONCLUDING COMMENTS
In this study we report on auditors’ perceptions on ADA use and the actual use of ADA on
109 different audits. The evidence was gathered from (1) semi-structured interviews with the
heads of professional practice of five international public accounting firms in Norway and (2)
responses to a detailed questionnaire from 216 engagement partners and/or managers on their
attitudes and perceptions towards the acceptance and application of ADA, and the
implementation of ADA on 109 audits of 2017 financial statements.
The results show that despite the firms’ global strategy to extend the ADA usage and the
auditors’ general positive attitude towards its use, the actual use of ADA is limited, and the use
33
Although not new (e.g., Arnold and Sutton 1998), there is a growing awareness within the academic literature
about limitations of the usage of ADA tools (e.g., Brown-Liburd et al. 2015; Appelbaum 2016; Sutton, Arnold, and
Holt 2018; No, Lee, Huang, and Lee 2019).
Downloaded
from
by
Aasmund
Eilifsen
on
15
June
2020

34
of more “advanced” ADA is rare. Some of the variation between firms in ADA use is explained
by differences in firm strategies in implementation of ADA. ADA output is mostly used as
supplementary evidence. As expected, more ADA are used for clients with integrated ERP/IT
systems. A higher frequency of ADA use is observed for new and tendered audit engagements.
The firms’ heads of professional practice express significant uncertainty about how the
supervisory inspection authorities will evaluate and accept ADA generated audit evidence. None
of the firms have introduced mandatory use of advanced ADA tools.
The study draws on institutional theory in the discussion of the findings. We conclude that
environmental pressures for firms to apply ADA primarily stem from technological
advancements and to some extent from audit clients, particularly new clients. Except for
promoting to external parties that they are engaged in the process of transforming to ADA usage
and to some extent towards audit clients, particularly new clients; legitimization of ADA usage
to other constituencies seems not to be a high priority for the surveyed firms. In deciding the use
of ADA at the engagement level, the auditor must be confident in the ability of the ADA tools to
efficiently and effectively provide sufficient and appropriate evidence. Institutionalized
conventions suggest that the limited use of ADA will likely persist until its use is incorporated
internally in the firms’ methodology and reward systems; explicitly supported and accepted by
standard-setters and supervisory bodies; and practitioners must find that ADA use proves
efficient and effective in the evidence gathering process.
The results from the study point to several potential fruitful avenues for future research. It
would be interesting to perform follow-up studies to learn (1) how the use of ADA in practice
develops over time and (2) whether ADA usage is more accelerated for audits of very large
companies than for audits in Norway, a relatively small country with limited number of large
Downloaded
from
by
Aasmund
Eilifsen
on
15
June
2020

35
companies. A key challenge for more ADA use seems to be how to cost-efficiently transform the
output from ADA to sufficient and appropriate audit evidence. Research can contribute to better
understand which factors inhibit this transformation to assist auditors to better apply ADA, for
example along the lines suggested by No et al. (2019). Uncertainty about supervisory bodies’
inspection behavior inhibits auditors’ use of ADA. A better understanding on how inspection
risk affects auditors’ ADA use (or none-use) is of importance. Finally, insight into how clients
may drive the demand for auditors’ ADA use, including the role of ADA in the tender
engagement process, seems a promising area for future research.
Downloaded
from
by
Aasmund
Eilifsen
on
15
June
2020

36
REFERENCES
Alles, M. G. 2015. Drivers of the use and facilitators and obstacles of the evolution of Big Data
by the audit profession. Accounting Horizons 29 (2): 439-449. DOI: 10.2308/acch-51067.
American Institute of Certified Public Accountants (AICPA). 2014. White Paper. Reimagining
auditing in a wired world. Available at:
https://www.aicpa.org/InterestAreas/FRC/AssuranceAdvisoryServices/DownloadableDocume
nts/Whitepaper_Blue_Sky_Scenario-Pinkbook.pdf
-----. 2015. Audit analytics and continuous audit: Looking toward the future. Available at:
https://www.aicpa.org/InterestAreas/FRC/AssuranceAdvisoryServices/DownloadableDocume
nts/AuditAnalytics_LookingTowardFuture.pdf
-----. 2017. AICPA Guide to audit data analytics.
Appelbaum, D., A. Kogan, and M. A. Vasarhelyi. 2017. Big Data and analytics in the modern
audit engagement: Research needs. Auditing: A Journal of Practice & Theory 36 (4): 1-27.
-----. 2018. Analytical procedures in external auditing: A comprehensive literature survey and
framework for external audit analytics. Journal of Accounting Literature 40: 83-101.
Appelbaum, D. 2016. Securing Big Data Provenance for Auditors: The Big Data provenance
black box as reliable evidence. Journal of Emerging Technologies in Accounting 13 (1): 17-
36.
Arnold, V., and S. G. Sutton. 1998. The theory of technology dominance: Understanding the
impact of intelligent decision aids on decision makers’ judgments. Advances in Accounting
Behavioral Research 1: 175-194.
Austin, A., T. Carpenter, M. Christ and C. Nielson. 2019. The data analytics transformation:
Evidence from auditors, CFOs, and standard-setters. Working paper
Barr-Pulliam, D., H. L. Brown-Liburd, and K. A. Sanderson. 2019. The Effects of the internal
control opinion and use of audit data analytics on perceptions of audit quality, assurance, and
auditor negligence. Working paper.
Bierstaker, J., D. Janvrin, and D. J. Lowe. 2014. What factors influence auditors' use of
computer-assisted audit techniques? Advances in Accounting 30: 67-74. DOI:
10.1016/j.adiac.2013.12.005.
Bloomfield, R., M. W. Nelson, and E. Soltes. 2016. Gathering data for archival, field, survey,
and experimental accounting research. Journal of Accounting Research 54 (2): 341-395.
Cao, M., R. Chychyla, and T. Stewart. 2015. Big Data analytics in financial statement audits.
Accounting Horizons 29 (2): 423-429.
Chartered Professional Accountants of Canada (CPA Canada). 2017. Audit data analytics alert.
Survey on use of audit data analytics in Canada - Results and possible implications. Available
at: https://www.cpacanada.ca/-/media/site/business-and-accounting-resources/docs/01453-rg-
audit-data-analytics-alert-survey-using-ada-in-canada-october-2017.pdf
Choi, J.-H., and T. J. Wong. 2007. Auditors’ governance functions and legal environment: An
international investigation. Contemporary Accounting Research 24: 13-46.
Downloaded
from
by
Aasmund
Eilifsen
on
15
June
2020

37
Cohn, M. 2016. Deloitte leverages AI and analytics for audits. Accounting Today November 14.
Available at: https://www.accountingtoday.com/news/deloitte-leverages-ai-and-analytics-for-
audits
Committee of European Audit Oversight Bodies (CEAOB). 2018. CEAOB work programme
2019. Available at: https://ec.europa.eu/info/business-economy-euro/banking-and-
finance/financial-reforms-and-their-progress/regulatory-process-financial-services/expert-
groups-comitology-and-other-committees/committee-european-auditing-oversight-
bodies_en#subgroups
Curtis, E., and S. Turley. 2007. The business risk audit – A longitudinal case study of an audit
engagement. Accounting, Organizations and Society 32 (4-5): 439-461.
Deephouse, D. L, J. Bundy, L. P. Tost, and M. C. Suchman. 2017. “Organizational legitimacy:
Six key questions” In R. Greenwood, C. Oliver. T. B. Lawrence, and R. E. Meyer (eds.) The
SAG Handbook of Organizational Institutionalism. Thousand Oaks, CA: Sage Publications,
Inc.: 27-54.
Deloitte. 2016. Perspectives. The power of advanced audit analytics: Bringing greater value to
external audit processes. Available at: https://www2.deloitte.com/us/en/pages/deloitte-
analytics/articles/us-the-power-of-advanced-audit-analytics.html
Digital Audit Research Project, Norwegian School of Economics (NHH). 2020.
https://www.nhh.no/en/research-centres/digaudit/
DiMaggio, P. J., and W. W. Powell. 1983. The iron cage revisited: Institutional isomorphism and
collective rationality in organizational fields. American Sociological Review 48 (2): 147-460.
-----. “Introduction” In P. L. DiMaggio and W. W. Powell (eds.) The New Institutionalism in
Organizational analyses. Chicago, IL: The University of Chicago Press: 1-40.
Dowling, C. 2009. Appropriate audit support system use: The influence of auditor, audit team,
and firm factors. The Accounting Review 84 (3): 771-810. DOI: 10.2308/accr.2009.84.3.771.
European Commission (EC). 2018. International digital economy and society index 2018.
Available at: https://ec.europa.eu/digital-single-market/en/news/international-digital-
economy-and-society-index-2018
EY. 2017a. How audit can benefit from a dive into deep data. Hermann Sidhu, EY Global
Assurance Digital Leader. Available at: https://www.ey.com/en_gl/assurance/how-audit-can-
benefit-from-a-dive-into-deep-data
-----. 2017b. How Big Data and analytics are transforming the audit. Available at:
https://www.ey.com/en_gl/assurance/how-big-data-and-analytics-are-transforming-the-audit
-----. 2019. Press release 18 January 2019. Available at:
https://www.ey.com/en_gl/news/2019/01/carmine-di-sibio-elected-as-next-ey-global-
chairman-and-ceo-effective-july
Financial Reporting Council (FRC). 2017. Audit quality thematic review: The use of data
analytics in the audit of financial statements. Available at:
https://www.frc.org.uk/getattachment/4fd19a18-1beb-4959-8737-
ae2dca80af67/AQTR_Audit-Data-Analytics-Jan-2017.pdf
Downloaded
from
by
Aasmund
Eilifsen
on
15
June
2020

38
Gartner. 2019. IT Glossary. Available at: https://www.gartner.com/it-glossary/
Geels, F. W. 2004. From sectoral systems of innovation to socio-technical systems: Insights
about dynamics and change from sociology and institutional theory. Research Policy 33 (6):
897-920.
Gepp, A., M. K. Linnenluecke, T. J. O'Neill, and T. Smith. 2018. Big Data techniques in auditing
research and practice: Current trends and future opportunities. Journal of Accounting
Literature 40: 102-115.
Goodhue, D., and R. Thompson. 1995. Task-technology fit and individual performance. MIS
Quarterly 19 (2): 213-233.
Greenwood, R., R. Suddaby, and C. R. Hinings. 2002. Theorizing change: The role of
professional associations in the transformation of institutionalized fields. Academy of
Management Journal 45 (1): 58-80.
Griffin, P. A., and A. M. Wright. 2015. Commentaries on Big Data’s importance for accounting
and auditing. Accounting Horizons 29 (2): 377-379.
Griffith, E. E., J. S. Hammersley, and K. Kadous. 2015. Audits of complex estimates as
verification of management numbers: How institutional pressures shape practice.
Contemporary Accounting Research 32 (3): 833-863.
Hope, O.-K., T. Kang, W. Thomas, and Y. K. Yoo. 2009. Impact of excess auditor remuneration
on the cost of equity capital around the world. Journal of Accounting, Auditing, and Finance
24 (2): 177-210.
International Auditing and Assurance Standards Board (IAASB). 2016. Data Analytics Working
Group. Exploring the growing use of technology in the audit, with a focus on data analytics.
Available at: http://www.ifac.org/publications-resources/exploring-growing-use-technology-
audit-focus-data-analytics
-----. 2019. Audit Evidence Working Group: Analysis of issues and proposed actions. Agenda
Item 7 at IAASB Meeting June 17-21, 2019. Available at:
http://www.iaasb.org/meetings/iaasb-board-meeting-new-york-usa
Institute of Chartered Accountants in England and Wales (ICAEW). 2016. Data Analytics for
external auditors. Available at: https://www.icaew.com/global-accounting-and-
auditing/global-auditing-perspectives/data-analytics-for-external-auditors
Instituut van de Bedrijfsrevisoren (IBR). 2018. Data analytics: The future of audit. Available at:
https://www.ibr-ire.be/nl/publicaties/actuele_reeksen/brochures/Pages/Data-Analytics.aspx
Kenno, S. A., S. A. McCracken, and S. E. Salterio. 2016. Financial reporting interview-based
research: A field research primer with an illustrative example. Behavioral Research in
Accounting 29 (1): 77-102.
KPMG. 2016. Data analytics and your audit. Available at:
https://home.kpmg/us/en/home/insights/2016/02/data-analytics-audit-article.html
Leuz, C., D. Nanda, and P. Wysocki. 2003. Earnings management and investor protection: an
international comparison. Journal of Financial Economics 69 (3): 505-27.
Li, H., J. Dai, T. Gershberg, and M. A. Vasarhelyi. 2018. Understanding usage and value of audit
Downloaded
from
by
Aasmund
Eilifsen
on
15
June
2020

39
analytics for internal auditors: An organizational approach. International Journal of
Accounting Information Systems 28: 59-76
Lillis, A. M. 1999. A framework for the analysis of interview data from multiple field research
sites. Accounting & Finance 39 (1): 79-105.
No, W. G., K. Lee, F. Huang, and Q. Li. 2019. Multidimensional audit data selection (MADS):
A framework for using data analytics in audit data selection process. Accounting Horizons 33
(3): 127-140.
O’Dwyer, B. 2010. What’s qualitative in accounting (field) research? An introduction to
qualitative field research in accounting at the 25th Contemporary Accounting Research
(CAR) Conference 2010.
O’Dwyer, B., D. Owen, and J. Unerman. 2011. Seeking legitimacy for new assurance forms: the
case of assurance on sustainability reporting. Accounting, Organizations and Society 36 (1):
31-52.
Payne, E. A., and M. B. Curtis. 2017. Factors associated with auditors’ intention to train on
optional technology. Current Issues in Auditing 11 (1): A1-A21. DOI: 10.2308/ciia-51564.
Perols, J. L., R. M. Bowen, C. Zimmermann, and B. Samba. 2017. Finding needles in a haystack:
Using data analytics to improve fraud prediction. The Accounting Review 92 (2): 221-245.
DOI: 10.2308/accr-51562
Power, M. 1997. The audit society: Rituals of verification. Oxford, UK: Oxford University Press.
-----. 2003. Auditing and the production of legitimacy. Accounting, Organizations and Society 28
(4): 379-94.
Power, M. K., and Y. Gendron. 2015. Qualitative research in auditing: A methodological
roadmap. Auditing: A Journal of Practice & Theory 34 (2): 147-165.
Public Company Accounting Oversight Board (PCAOB). 2018. Strategic plan 2018-2022.
Available at: https://pcaobus.org/Pages/PCAOB-Strategic-Plan-2018-2022.aspx
PwC. 2017. Presentation of PwC U.S. Chief Auditor and Leader of Auditing Services,
Methodology & Tools. Leonard Combs at the PCAOB Standing Advisory Group Meeting
May 24-25, 2017. Available at: https://pcaobus.org/News/Events/Pages/SAG-meeting-May-
2017.aspx
Robson, K., C. Humphrey, R. Khalifa, and J. Jones. 2007. Transforming audit technologies:
Business Risk Audit methodologies and the audit field. Accounting, Organizations and
Society 32 (4-5): 409-438.
Salijeni, G., A. Samsonova-Tadderu, and S. Turley. 2019. Big Data and changes in audit
technology: contemplating a research agenda. Accounting and Business Research 49 (1): 95-
119.
Schneider, G., J. Dai, D. Janvrin, K. Ajayi, and R. L. Raschke. 2015. Infer, predict, and assure:
Accounting opportunities in data analytics. Accounting Horizons 29 (3): 719-42.
doi:10.2308/acch-51140
Suchman, M. C. 1995. Managing legitimacy: Strategic and institutional approaches. Academy of
Management Review 20 (3): 571-610.
Downloaded
from
by
Aasmund
Eilifsen
on
15
June
2020

40
Sutton, S., V. Arnold, and M. Holt. 2018. How much automation is too much? Keeping the
human relevant in knowledge work. Journal of Emerging Technologies in Accounting 15 (2):
15-25.
Walker, K. R., and H. Brown-Liburd. 2019. The Emergence of data analytics in auditing:
Perspectives from internal and external auditors through the lens of institutional theory.
Working paper May 2019.
Weick, K. E., K. M. Sutcliffe, and D. Obstfeld. 2005. Organizing and the process of
sensemaking. Organizational Science 16 (4): 409-421.
Wooten, M., and A. J. Hoffman. 2017. “Organizational legitimacy: Six key questions” In R.
Greenwood, C. Oliver. T. B. Lawrence, and R. E. Meyer (eds.) The SAG Handbook of
Organizational Institutionalism. Thousand Oaks, CA: Sage Publications, Inc.: 55-74.
Zhou, T., Y. Lu, and B. Wang. 2010. Integrating TTF and UTAUT to explain mobile banking
user adoption. Computers in Human Behavior 26: 760-767.
Downloaded
from
by
Aasmund
Eilifsen
on
15
June
2020

41
APPENDIX A
Interview Script – Heads of Professional Practice
Estimated time: 60-90 min
Introduction
Background data:
Personal data/firm
Give an overview of the current type of data tools and data analytics (CAAT/ADA).
Type of tools in the firm, how much is in use or implemented in Norway.
Now, please give specific comments to the status on CAAT and ADA on the different phases of
the audit:
1. Client acceptance or renewal phase
Does your firm have standardized data tools with interactive yes/no answers, scoring systems or
is your firm using internal/external data bases that provides guidance or answers on whether an
audit client can be accepted or renewed? Please describe.
Developed globally or locally/nationally? Is the system mandatory or voluntary?
2. Fraud / Fraud (ISA 240)
Does your firm have standardized data tools for assessing fraud? What is the name of the system
and explain in short how it works, and what sources (external/internal) it uses and how the
results are generated?
Developed globally or locally/nationally? Is the system mandatory? For all clients / some type of
clients (listed/SEC/all major)?
3. Planning/Risk Assessment
Does you firm have standardized data tools or standardized data analysis according to the audit
firm's audit methodology in the planning phase for risk identification, assessment of internal
control, audit plan, etc.?
Examples in different phases of the audit if the Head does not answer without support:
- For use in business understanding; analysis of past accounts, market understanding, trends, etc.
- Assessment of control environment, internal control, IT environment
- Other types of 'audit planning software' or 'risk assessment software'
- The questions do not apply to project management, hourly usage, fees or etc. We will return to
material calculations.
For each of these systems:
What are the name of the tools and explain in short, how they work and what sources
(external/internal) they use and how results are generated.
Developed globally or locally/nationally? Is the system mandatory? For all clients/some type of
clients (listed/SEC/all)?
Other types of data analysis (voluntary or mandatory) included in the audit company's standard
methodology during the planning phase and risk assessment?
4. Materiality:
Does your firm have standardized data tools or standardized data analysis according to the audit
firm's audit methodology in the planning phase for calculating and distributing/detailing
Downloaded
from
by
Aasmund
Eilifsen
on
15
June
2020

42
materiality into single areas and single entities?
What is the name of the system and explain in short, how it works and what sources
(external/internal) it uses and how the results are generated.
Developed globally or locally/nationally? Is the system mandatory? For all clients/some type of
5. Mapping of IT environments and internal control tests
firm's audit methodology for mapping IT environments, internal control and internal control
testing?
- Mapping of transaction processes and/or controls
- Mapping of IT environments and systems
- Testing of IT controls, databases
- Testing of transactions, monitoring of transactions
- Other
For each of these tools or analysis:
What are the systems named and explain in short, how it works and what sources
(external/internal) it uses and how results are generated.
Developed globally or locally / nationally? Is the system mandatory? For all clients/some type of
Other types of data analysis (voluntary or mandatory) included in the audit company's
standardized method of testing internal control?
6. Substantive testing / detail testing
firm's audit methodology for substantive testing?
- Sampling/random selection tools
- Tools for summation control/completeness/accuracy etc.
- For example, Journal Entry testing
- Summary tools for testing accuracy in different files
- Calculation controls
- XL spreadsheet formula checks, summaries and similar
Data mining tools;
- Text Analysis
- Search for special numbers, transactions with special characteristics (negative numbers, all over
X in value and similar)
- Test of entire populations instead of variety testing/sampling
Analytical tests
- Analytical tools; for example, for use of revenue testing
- Analytical tools; for example, for stock inventory testing
- Analytical tools; for example, for the use of claims testing
- Analytical tools; for example, for use on taxes, fees, holiday pay, VAT testing
- Analytical tools; for example, for testing of other costs and provisions
Downloaded
from
by
Aasmund
Eilifsen
on
15
June
2020

43
Tools or data analysis methods for auditing estimates or value estimates
- Valuation of derivatives
- Valuation for testing of impairment
- Other analysis or tests of estimates
- Visualization tool with standardized presentations
- Tools for use for external confirmations
- Tools for confirmation of existence e.g., of inventories; camera, drones and similar
Other types of data analysis (voluntary or mandatory) included in the audit company's standard
methodology in substantive tests/detail testing?
For each of these tools and analysis:
What are the systems named and explain in short, how it works and what sources
(external/internal) it uses and how results are generated. Is it excel based or based on other
software?
To what extent are 'simple' analysis (deviations from average, abnormal sizes) or more statistical
analysis, clustering or sorting/structuring large amounts of unstructured data?
Are analysis or tools used for risk identification or audit evidence?
Developed globally or locally/nationally?
Is the system mandatory? For all clients/some type of clients (listed/SEC/all)?
7. Wrap up/summary / conclusion
firm's audit methodology for use in summary and conclusion?
For example:
- Reconciliation/control between final accounts and underlying ledgers etc.
- Text control, control of numbers and text compared to last year´s, internal consistency and
similar.
- Analytical checks/fairness checks of results, balance values, etc.
- Subsequent/events post balance sheet date
About actual use of data tools in Norway
Actual implementation of audit assignments carried out in Norway by the various tools:
-Has the firm or the Norwegian firm taken any formal position of the use of data analytics and
related tools in general or in special areas? Any impediments in your opinion on use vs. GAAS
or supervisory authorities?
- In your opinion, to what extent has the various tools been implemented in audit assignments in
Norway for the 2017 audit?
- Differences between SEC, listed, major/minor audit assignments, industries?
- Differences depending on customer characteristics (IT systems, degree of integrated
systems, own data analysis, expectations/attitudes)?
- Selected pilots for general use? Specific tools pilot tested on some type of audit assignments?
What types and explain why.
- Degree of mandatory use/voluntary/incentives (bonus/'allowed' to spend several hours) /
measurement/expectation and central pressure on use?
- Type and amount of training, support etc.
- Your assessment of user complexity/user threshold, implementation costs, efficiency and/or
improvement of audit quality for the various tools. You may be able to provide some tools that
you think have low user threshold, low implementation costs, high efficiency gains, or quality
Downloaded
from
by
Aasmund
Eilifsen
on
15
June
2020

44
improvement, and vice versa.
- Or other factors (but not personal characteristics of the team/responsible partner) that affect the
extent of the use of data tools and data analysis? In your opinion, what may prevent more
extensive use?
- What do you think about the use of tools and data analysis, for example comparing two to three
years ahead from today?
Other comments?
Thank you!
Downloaded
from
by
Aasmund
Eilifsen
on
15
June
2020

45
Table 1
Engagements per Audit Firm
Firm
Category of engagements1
ADA Listed Large_NL Small_NL Total
1 6 6 6 6 24
2 6 5 5 4 20
3 6 6 5 6 23
4 5 5 5 4 19
5 5 6 6 6 23
Totals 28 28 27 26 109
Notes:
1
ADA: Engagement entity where the audit firm expected high ADA use. Of the 28 ADA engagements, 8 were listed
(4 of 8 were newly tendered audits) and 20 were non-listed. Of the non-listed companies, 16 were “Large” and 4
were “Small.”
Listed: Largest engagement entity (a parent company or a subsidiary) based on revenues of a listed company
(group).
Large_NL: Largest engagement entity (a parent company or a subsidiary) based on revenues of a non-listed (NL)
group with group revenues above $100 million.
Small_NL: Engagement entity of a non-listed (NL) group with group revenues below $10 million, or a non-listed
(NL) entity not part of a group with revenues below $10 million.
Downloaded
from
by
Aasmund
Eilifsen
on
15
June
2020

46
Table 2
Engagement Statistics
Panel A: Industries [number (%)]
N
Category of Engagement1
Industry ADA Listed Large_NL Small_NL
Agriculture, forestry, fishing 6 2 (33.3) 0 (0.0) 4 (66.7) 0 (0.0)
Construction 6 0 (0.0) 3 (50.0) 1 (16.7) 2 (33.3)
Entertainment, leisure 1 1 (100.0) 0 (0.0) 0 (0.0) 0 (0.0)
Financial services 6 3 (50.0) 2 (33.3) 0 (0.0) 1 (16.7)
Holding company (in a group), investment
entity, conglomerates
7 0 (0.0) 3 (42.9) 0 (0.0) 4 (57.1)
Manufacturing 10 4 (40.0) 4 (40.0) 2 (20.0) 0 (0.0)
Media, communications, publishing 1 0 (0.0) 0 (0.0) 1 (100.0) 0 (0.0)
Not for profit 2 1 (50.0) 0 (0.0) 1 (50.0) 0 (0.0)
Hospital, health organizations, social services 4 0 (0.0) 0 (0.0) 4 (100.0) 0 (0.0)
Oil & gas services 3 0 (0.0) 2 (66.7) 0 (0.0) 1 (33.3)
Pharmaceuticals and chemicals 3 0 (0.0) 1 (33.3) 1 (33.3) 1 (33.3)
Real estate 6 1 (16.7) 2 (33.3) 1 (16.7) 2 (33.3)
Retail, wholesale 19 8 (42.1) 2 (10.5) 3 (15.8) 6 (31.6)
Software 7 3 (42.9) 0 (0.0) 0 (0.0) 4 (57.1)
Telecommunications 4 1 (25.0) 2 (50.0) 1 (25.0) 0 (0.0)
Transportation, shipping, distribution 13 2 (15.4) 5 (38.5) 5 (38.5) 1 (7.7)
Utilities 3 0 (0.0) 1 (33.3) 1 (33.3) 1 (33.3)
Other 8 2 (25.0) 1 (12.5) 2 (25.0) 3 (37.5)
Total 109 28 (25.7) 28 (25.7) 27 (24.7) 26 (23.9)
Panel B: Size and Group Affiliation [number (%)]
N
Total Revenues (2017) ADA Listed Large_NL Small_NL
Less than $10 million 37 2 (5.4) 9 (24.3) 1 (2.7) 25 (67,6)
$10 - $100 million 19 9 (47.3) 8 (42.1) 1 (5.3) 1 (5.3)
$100 million - $1 billion 36 11 (30.6) 7 (19.4) 18 (50.0) 0 (0.0)
Greater than $1 billion 17 6 (35.3) 4 (23.5) 7 (41.2) 0 (0.0)
Total 109 28 28 27 26
Group Affiliation
Entity not part of a group 14 3 (21.4) 0 (0.0) 0 (0.0) 11 (78.6)
Parent company of a group 37 8 (21.6) 19 (51.4) 8 (21.6) 2 (5.4)
Subsidiary of a Norwegian parent company 32 11 (34.4) 4 (12.5) 10 (31.3) 7 (21.9)
Subsidiary of a non-Norwegian parent company 26 6 (23.1) 5 (19.2) 9 (34.6) 6 (23.1)
Total 109 28 28 27 26
Downloaded
from
by
Aasmund
Eilifsen
on
15
June
2020

47
Panel C: Audit Hours
Notes:
1
ADA: Engagement entity where the audit firm expected high ADA use.
Listed: Largest engagement entity (a parent company or a subsidiary) based on revenues of a listed company
(group).
Large_NL: Largest engagement entity (a parent company or a subsidiary) based on revenues of a non-listed (NL)
group with group revenues above $100 million.
Small_NL: Engagement entity of a non-listed (NL) group with group revenues below $10 million, or a non-listed
2
One engagement failed to report audit hours.
Overall ADA Listed Large_NL Small_NL
Average audit hours per engagement
(range) [SD] (n=108)2
891 1575
(50-12,000)
[2575]
936
(100-2500)
[1457]
958
(150-7,000)
[675]
95
(10-500)
[105]
Downloaded
from
by
Aasmund
Eilifsen
on
15
June
2020

48
Table 3
Participant Demographics
(n=216)
Gender Number Percent
Male 177 82
Female 39 18
Age
Under 30 17 8
30-39 88 41
40-49 70 32
50-59 36 17
60 or older 5 2
Education
Bachelor’s in accounting and auditing 18 8
Master’s in accounting and auditing, master’s in business administration
or similar1
198 92
Years of Audit Experience
Less than 5 years 21 10
5-10 years 54 25
11-15 years 41 19
16-20 years 26 12
More than 20 years 74 34
Position within your firm
Partner 103 48
Director 24 11
Senior manager 22 10
Manager 36 17
Senior associates 22 10
Other (supervisor, associate, staff) 9 4
Average percentage of participants’ time allocated to types of audit clients Percent
Listed company 34
Governmentally owned (non-listed) 4
Non-listed large entities (greater than $100 million in revenues) 23
Non-listed medium entities ($10 million to $100 million in revenues) 21
Non-listed small entities (less than $10 million in revenues) 16
Other 2
100
Notes:
1
Ninety-seven hold two master’s degrees; one master’s in accounting and auditing plus one master’s in business
administration or similar.
Downloaded
from
by
Aasmund
Eilifsen
on
15
June
2020

49
Table 4
Participants’ Perceived Competence with IT Systems and ADA Tools
(n=216)
Average Ratings1
Perceived competence with: Overall Partner Manager
IT systems and ERP systems 4.09 3.82* 4.27*, #
Data programming, data handling, and data storage 3.43* 3.27* 3.60*
Interpreting the results from data analytics in general 4.80* 4.70* 4.89*
ADA tools which are available to you in your firm 4.46* 4.14 4.77*, #
Notes:
1
The rating scale is 1 – Very Low to 7 – Very High.
* p<0.05 (one-tailed). Mean ratings are tested for significant difference (greater/less) from the midpoint (4) of the
rating scale.
#
Significant at p<0.05 (two-tailed). Mean ratings are tested for significant difference between partners and
managers.
Downloaded
from
by
Aasmund
Eilifsen
on
15
June
2020

50
Table 5
Perceptions of Competency and Usefulness of ADA
(n=216)
Panel A: Perceived Competency of ADA
Overall
Mean1
I have sufficient knowledge of ADA tools and procedures available in our firm to use them
appropriately.
4.42*
I have been given sufficient training of usage of ADA tools available in our firm. 4.08
I have sufficient knowledge of the ADA statistical methods and their interpretation. 4.20
I have been given sufficient training of the usage of ADA statistical methods and their interpretation. 4.00
Panel B: Perceived Usefulness of ADA for Audits in General within Their Firm
The ADA tools that I have seen, tested or used in our firm are complex and difficult to master and
apply in audits.
3.67*
The ADA tools that I have seen, tested or used in our firm are very time-consuming to implement on
my engagements.
4.07
Our audit firm gives incentives and/or exerts pressure to use ADA instead of traditional audit tools
and procedures.
4.88*
Current auditing standards accept the use of audit evidence derived from ADA. 4.54*
Our firm’s audit methodology accepts the use of audit evidence derived from ADA. 5.33*
Supervisory bodies inspecting audit engagements accept audit evidence derived from ADA. 4.59*
In my opinion. ADA are currently useful for risk identification. 5.27*
In my opinion. ADA are currently useful for substantive testing to obtain audit evidence. 5.13*
I am personally hesitant to use ADA with the ADA tools and procedures currently available in our
firm.
2.93*
Our firm has adequate ADA tools. 4.85*
Compared to traditional audit tools and procedures, the ADA tools and procedures currently
available in our firm are improving the quality of audits.
5.44*
Compared to traditional audit tools and procedures, the ADA tools and procedures currently
available in our firm are efficient (time. effort or cost in generating the result) to use in audits.
4.40*
In our firm, we lack technical competence on the audit teams to use ADA tools. 3.70*
In our firm, we have sufficient specialist support (outside the core teams) for data-gathering,
handling and/or ADA when needed on engagements.
5.15*
Our firm’s clients have a positive attitude to our collection of client data for ADA purposes. 5.19*
It is easy to collect sufficient data for ADA purposes from our firm’s clients. 3.86
Panel C: Perceived Usefulness of ADA for the Selected Engagements
In my opinion. ADA are currently useful for risk identification on this engagement. 4.62*
In my opinion. ADA are currently useful for substantive testing to obtain audit evidence on this
engagement.
4.68*
I am personally hesitant to use ADA in this engagement with the ADA tools and procedures currently
available in our firm.
2.79*
Our firm has adequate ADA tools for this engagement. 4.79*
Compared to traditional audit tools and procedures, the ADA tools and procedures currently available
in our firm are improving the quality on this engagement.
4.85*
Compared to traditional audit tools and procedures, the ADA tools and procedures currently available
in our firm are efficient (time. effort or cost in generating the result) to use on this engagement.
4.30*
We lack technical competence on the audit teams to use ADA tools on this engagement. 3.07*
We have sufficient specialist support (outside the core teams) for data-gathering, handling and/or ADA
when needed on this engagement.
4.62*
On this engagement, the client has a positive attitude to our collection of client data for ADA purposes. 4.68*
Downloaded
from
by
Aasmund
Eilifsen
on
15
June
2020

51
It is easy to collect sufficient data for ADA purposes from the client on this engagement. 3.99
Notes:
1
Average score by rating scale: 1-Strongly disagree, 4-Neutral, 7-Strongly agree.
* p<0.05 (one-tailed). Mean ratings are tested for significant difference (greater/less) from the midpoint (4) of the
rating scale.
Downloaded
from
by
Aasmund
Eilifsen
on
15
June
2020

52
Table 6
Client IT and ADA Data Collection
Panel A: What best describes the entity's ERP/IT system related to accounting?
Type of Engagement1
N
(%)
ADA Listed Large_NL Small_NL
General ledger is not or to a limited extent
integrated with other accounting systems.
11
(10.1)
1
(9.1)
1
(9.1)
3
(27.3)
6
(54.5)
The accounting system is partially integrated. 41
(37.6)
8
(19.5)
13
(31.7)
12
(29.3)
8
(19.5)
The accounting system is highly integrated/highly
integrated with other systems.
45
(41.3)
19
(42.2)
10
(22.2)
11
(24.4)
5
(11.1)
The accounting system is outsourced, and is not
3
(2.8)
0
(0.0)
2
(66.7)
0
(0.0)
1
(33.3)
The accounting system is outsourced, and is highly
9
(8.3)
0
(0.0)
2
(22.2)
1
(11.1)
6
(66.7)
Total 109 28 28 27 26
Panel B: Accounts or records where data were collected for ADA use
N2
General ledger 80 25
(31.3)
21
(26.3)
23
(28.8)
11
(13.8)
Accounts Payables 26 7
(26.9)
6
(23.1)
7
(26.9)
6
(23.1)
Account Receivables 25 8
(32.0)
4
(16.0)
7
(28.0)
6
(24.0)
Inventory 12 3
(25.0)
2
(16.7)
6
(50.0)
1
(8.3)
Fixed asset register 11 4
(36.4)
2
(18.2)
4
(36.4)
1
(9.1)
Payment-systems/bank account-ledgers 12 2
(16.7)
2
(16.7)
6
(50.0)
2
(16.7)
Salary 11 2
(18.2)
3
(27.3)
2
(18.2)
4
(36.4)
Financial investments 2 1
(50.0)
1
(50.0)
0
(0.0)
0
(0.0)
Other 11 7
(63.6)
2
(18.2)
2
(18.2)
0
(0.0)
Total 190 59 43 57 31
Panel C: Who collected the data for ADA use?
N
(%)
ADA Listed Large_NL Small-NL
Member(s) of the regular audit team. 17
(21.0)
2
(11.8)
6
(35.3)
6
(35.3)
3
(17.7)
IT specialists/a special department/specialist group
in the audit firm.
26
(32.1)
10
(38.5)
7
(26.9)
7
(26.9)
2
(7.7)
The client did most of the data collection work
(based on instructions from the auditor).
35
(43.2)
12
(34.3)
8
(22.9)
10
(28.6)
5
(14.3)
Downloaded
from
by
Aasmund
Eilifsen
on
15
June
2020

53
Others (external service providers etc.). 3
(3.7)
2
(66.7)
0
(0.0)
0
(0.0)
1
(33.3)
Total 81 26 21 23 11
Panel D: Who cleaned/structured the data for ADA use?
N
(%)
Member(s) of the regular audit team. 30
(37.1)
8
(26.7)
10
(33.3)
6
(20.0)
6
(20.0)
IT specialists/a special department/specialist group
in the audit firm.
47
(58.0)
18
(38.3)
8
(17.0)
16
(34.0)
5
(10.6)
The client did most of the ADA analysis (based on
instructions from the auditor).
1
(1.2)
0
(0.0)
0
(0.0)
1
(100.0)
0
(0.0)
Others (external service providers). 3
(3.7)
0
(0.0)
3
(100.0)
0
(0.0)
0
(0.0)
Total 81 26 21 23 11
Panel E: Who conducted the main part of the ADA?
N
(%)
Member(s) on the regular audit team. 56
(69.1)
18
(32.1)
14
(25.0)
16
(28.6)
8
(14.3)
IT specialists (a special department/specialist group
in the audit firm.
24
(29.7)
8
(33.3)
6
(25.0)
7
(29.2)
3
(12.5)
The client did most of the ADA analysis (based on
instructions from the auditor).
0
(0.0)
0
(0.0)
0
(0.0)
0
(0.0)
0
(0.0)
Others (external service providers etc.). 1
(1.2)
0
(0.0)
1
(100.0)
0
(0.0)
0
(0.0)
Total 81 26 21 23 11
Notes:
1
ADA: Engagement entity where the audit firm expected high ADA usage.
Listed: Largest engagement entity (parent company or subsidiary) based on revenues of a listed company (group).
Large_NL: Largest engagement entity (parent company or subsidiary) based on revenues of a non-listed (NL) group
with group revenues above $100 million.
Small_NL: Engagement entity of a non-listed (NL) group with group revenues below $10 million or a non-listed
2
The total is greater than the 81 engagements reported ADA use because ADA were used in multiple accounts or
records.
Downloaded
from
by
Aasmund
Eilifsen
on
15
June
2020

54
Table 7
Use of ADA in the Phases of the Audit
Audit Firm
ADA Usage by Audit Phases and Audit Steps N2
ADA Listed Large_NL Small_NL 1 2 3 4 5
Audit Phases Audit Steps
Client
Acceptance and
Continuance
25 7 (28.0) 3 (12.0) 8 (32.0) 7 (28.0) 6 (24.0) 4 (16.0) 11 (44.0) 0 (0.0) 4 (16.0)
Fraud Risk
Assessment
19 7 (36.8) 2 (10.5) 5 (26.3) 5 (26.3) 7 (36.8) 2 (10.5) 4 (21.1) 2 (10.5) 4 (21.1)
Planning Overall assessment of industry risks, understanding the
business
17 6 (35.3) 2 (11.8) 3 (17.7) 6 (35.3) 7 (41.2) 4 (23.5) 3 (17.7) 2 (11.8) 1 (5.9)
Overall assessment of client's operations and
performance
33 8 (24.2) 10 (30.3) 7 (21.2) 8 (24.2) 16 (48.5) 6 (18.2) 3 (9.1) 6 (18.2) 2 (6.1)
Assessing going concern issues (score, rating etc.) 11 2 (18.2) 3 (27.3) 2 (18.2) 4 (36.4) 6 (54.6) 2 (18.2) 1 (9.1) 0 (0.0) 2 (18.2)
Mapping of different processes 26 9 (34.6) 4 (15.4) 7 (26.9) 6 (23.1) 13 (50.0) 4 (15.4) 6 (23.1) 2 (7.7) 1 (3.9)
Identifying, assessing key risks 27 9 (33.3) 5 (18.5) 7 (25.9) 6 (22.2) 11 (40.7) 4 (14.8) 3 (11.1) 5 (18.5) 4 (14.8)
Determining and allocating materiality 18 5 (27.8) 5 (27.8) 2 (11.1) 6 (33.3) 11 (61.1) 2 (11.1) 2 (11.1) 2 (11.1) 1 (5.6)
Other planning 4 2 (50.0) 0 (0.0) 2 (50.0) 0 (0.0) 1 (25.0) 0 (0.0) 1 (25.0) 0 (0.0) 2 (50.0)
Internal
Control
Test of controls using extracted client data 37 17 (46.0) 9 (24.3) 7 (18.9) 4 (10.8) 9 (24.3) 7 (18.9) 9 (24.3) 5 (13.5) 7 (18.9)
Other internal controls 11 5 (45.5) 3 (27.3) 2 (18.2) 1 (9.1) 2 (18.2) 3 (27.3) 1 (9.1) 4 (36.4) 1 (9.1)
Substantive
Testing
Journal entry testing 74 24 (32.4) 21 (28.4) 17 (23.0) 12 (16.2) 22 (29.7) 12 (16.2) 17 (23.0) 9 (12.2) 14 (18.9)
Calculating sample-size for testing 55 14 (25.5) 16 (29.1) 16 (29.1) 9 (16.4) 16 (29.1) 10 (18.2) 6 (10.9) 9 (16.4) 14 (25.5)
Selecting of random samples from population 51 17 (33.3) 12 (23.5) 14 (27.5) 8 (15.7) 16 (31.4) 10 (19.6) 1 (2.0) 9 (17.7) 15 (29.4)
Simple judgmental methods: deviations from average,
larger/smaller than/negative
40 13 (32.5) 10 (25.0) 11 (27.5) 6 (15.0) 16 (40.0) 9 (22.5) 5 (12.5) 3 (7.5) 7 (17.5)
Visual (non-statistical) identification of deviations,
outliers, groups or similar
37 16 (43.2) 9 (24.3) 8 (21.6) 4 (10.8) 11 (29.7) 9 (24.3) 8 (21.6) 5 (13.5) 4 (10.8)
Statistical regression 9 3 (33.3) 2 (22.2) 2 (22.2) 2 (22.2) 3 (33.3) 3 (33.3) 0 (0.0) 3 (33.3) 0 (0.0)
Statistical grouping, decision trees, clustering
techniques by the computer tool
10 4 (40.0) 3 (30.0) 1 (10.0) 2 (20.0) 4 (40.0) 3 (30.0) 3 (30.0) 0 (0.0) 0 (0.0)
Making statistical calculations from the data for
predictive analysis
21 10 (47.6) 3 (14.3) 4 (19.1) 4 (19.1) 9 (42.9) 3 (14.3) 4 (19.1) 3 (14.3) 2 (9.5)
Text mining (in contracts, reports etc.) to identify
patterns, exceptions etc.
3 1 (33.3) 1 (33.3) 1 (33.3) 0 (0.0) 1 (33.3) 0 (0.0) 0 (0.0) 0 (0.0) 2 (66.7)
2 or 3 way matching of specific amounts (value, # of
items etc.)
36 16 (44.4) 5 (13.9) 11 (30.6) 4 (11.1) 16 (44.4) 11 (30.6) 7 (19.4) 1 (2.8) 1 (2.8)
Other substantive analyses 4 2 (50.0) 1 (25.0) 1 (25.0) 0 (0.0) 2 (50.0) 0 (0.0) 1 (25.0) 1 (25.0) 0 (0.0)
Downloaded
from
by
Aasmund
Eilifsen
on
15
June
2020

55
Summarizing ledgers by computer audit tools 41 15 (36.6) 9 (22.0) 7 (17.1) 10 (24.4) 16 (39.0) 11 (26.8) 5 (12.2) 1 (2.4) 8 (19.5)
Recalculating, re-performing, testing formula (e.g. in
client excel spreadsheets) by computer audit tools
27 10 (37.0) 6 (22.) 6 (22.2) 5 (18.5) 10 (37.0) 4 (14.8) 2 (7.4) 5 (18.5) 6 (22.2)
Reconciliations; match total numbers between different
ledgers or similar
33 10 (30.3) 7 (21.2) 8 (24.2) 8 (24.2) 16 (48.5) 10 (30.3) 3 (9.1) 0 (0.0) 4 (12.1)
Automated reconciliation of external bank
confirmations versus clients data
1 1 (100.0) 0 (0.0) 0 (0.0) 0 (0.0) 0 (0.0) 0 (0.0) 0 0 (0.0) 1 (100.0) 0 (0.0)
Other automated reconciliations 1 1 (100.0) 0 (0.0) 0 (0.0) 0 (0.0) 0 (0.0) 0 (0.0) 1 (100.0) 0 0 (0.0) 0 (0.0)
Testing of valuation of financial instruments 6 2 (33.3) 2 (33.3) 1 (16.7) 1 (16.7) 3 (50.0) 0 (0.0) 0 (0.0) 2 (33.3) 1 (16.7)
Other computer assisted analytical tests of
impairment/estimates/values
5 1 (20.0) 3 (60.) 0 (0.0) 1 (20.0) 3 (60.0) 0 (0.0) 1 (20.0) 1 (20.0) 0 (0.0)
Other substantive testing as audit evidence 4 1 (25.0) 2 (50.0) 1 (25.0) 0 (0.0) 0 (0.0) 3 (75.0) 1 (25.0) 0 (0.0) 0 (0.0)
Final Review
and Conclusion
Review of board meetings, audit meetings or similar 1 1 (100.0) 0 (0.0) 0 (0.0) 0 (0.0) 0 (0.0) 1 (100.) 0 (0.0) 0 (0.0) 0 (0.0)
Analytical checks/fairness checks of results, balance
values etc.
25 10 (40.0) 6 (24.) 4 (16.0) 5 (20.0) 10 (40.0) 4 (16.0) 3 (12.0) 4 (16.0) 4 (16.0)
Reconciliation/control between final accounts and
underlying ledger etc.
26 9 (34.6) 6 (23.1) 5 (19.2) 6 (23.1) 9 (34.6) 7 (26.9) 2 (7.7) 1 (3.9) 7 (26.9)
Examining financial statement disclosures and notes 4 2 (50.0) 1 (25.0) 1 (25.0) 0 (0.0) 1 (25.0) 0 (0.0) 1 (25.0) 0 (0.0) 2 (50.0)
Control of numbers and text against last year's
financial report, internal consistency in current year's
financial
8 2 (25.0) 3 (37.5) 1 (12.5) 2 (25.0) 2 (25.0) 1 (12.5) 0 (0.0) 1 (12.5) 4 (50.0)
Subsequent/events after the balance sheet date 6 3 (50.0) 1 (16.7) 0 (0.0) 2 (33.3) 2 (33.3) 1 (16.7) 1 (16.7) 0 (0.0) 2 (33.3)
Using data analytics when forming the audit opinion 9 4 (44.4) 0 (0.0) 2 (22.2) 3 (33.3) 4 (44.4) 3 (33.3) 1 (11.1) 0 (0.0) 1 (11.1)
Other wrap up 1 0 (0.0) 1 (100.0) 0 (0.0) 0 (0.0) 0 (0.0) 1 (100.0) 0 (0.0) 0 (0.0) 0 (0.0)
Total 766 269 176 174 147 281 154 117 87 127
Average number of times ADA were used per engagement 7.0 9.6 6.3 6.4 5.7 11.7 7.7 5.1 4.6 5.5
Notes:
1
Large_NL: Largest engagement entity (parent company or subsidiary) based on revenues of a non-listed (NL) group with group revenues above $100 million.
Small_NL: Engagement entity of a non-listed (NL) group with group revenues below $10 million or a non-listed (NL) entity not part of a group with revenues
below $10 million.
2
Number of engagements with answer ‘yes’ to the question that ADA were used in the specific sub-category on the selected engagement out of the 109
engagements.
Downloaded
from
by
Aasmund
Eilifsen
on
15
June
2020

56
Table 8
Type of ADA Tools Used
Audit Firm
Which ADA tool was used2
Total3
ADA Listed Large_NL Small_NL 1 2 3 4 5
Excel-basic functionality 424 (34.4) 101 (23.8) 98 (23.1) 111 (26.2) 114 (26.9) 159 (37.5) 75 (17.7) 40 (9.4) 53 (12.5) 97 (22.9)
Excel-advanced 319 (25.9) 141 (44.2) 47 (14.7) 58(18.2) 73 (22.9) 144 (45.1) 66 (20.7) 61(19.1) 18 (5.6) 30 (9.4)
Analytical tools BI 85 (6.9) 21 (23.6) 22 (25.9) 37 (43.5) 5 (6.9) 19 (22.4) 20 (23.5) 36 (42.4) 10 (11.8) 0 (0.0)
Statistical packages 34 (2.7) 5 (14.7) 10 (29.4) 16 (47.1) 3 (8.8) 15 (44.1) 9 (26.5) 5 (14.7) 5 (14.7) 0 (0.0)
Database management systems 113 (9.2)) 53 (46.9) 26 (23.0) 23 (20.4) 11 (9.7) 69 (61.0) 23 (20.4) 6 (5.3) 1 (0.9) 14 (12.5)
Programming languages 27 (2.2) 3 (11.1) 4 (14.8) 1 (3.7) 19 (70.4) 23 (85.2) 1 (3.7) 2 (7.4) 0 (0.0) 1 (3.7)
Visualizations programs 130 (10.6) 68 (52.3) 33 (25.4) 29 (22.3) 0 (0.0) 51 (39.2) 10 (7.7) 11 (8.5) 32 (24.6) 26 (20.0)
Other 100 (8.1) 49 (49.0) 23 (23.0) 24 (24.0) 4 (4.0) 9 (9.0) 41 (41.0) 28 (28.0) 17 (17.0) 5 (5.0)
# of times ADA tools were used 1232 (100) 441 (35.8) 263 (21.3) 299 (24.3) 229 (18.9) 489 (39.7) 245 (19.9) 189 (15.3) 136 (11.0) 173 (14.0)
Notes:
1
Large_NL: Largest engagement entity (parent company or subsidiary) based on revenues of a non-listed (NL) group with group revenues above $100 million.
Small_NL: Engagement entity of a non-listed (NL) group with group revenues below $10 million or a non-listed (NL) entity not part of a group with revenues
below $10 million.
2
Excel- basic functionality, Excel- Advanced (e.g., Power-BI, VBA, Solver, Miner, Macros), Analytics tools BI (e.g., IBM, Oracle, SAP, SAS), Statistical
packages (e.g., ‘R’, Stata, SPSS, Matlab, SAS, JPM, Crystal Ball), Database management systems (e.g., Access, MySQL, SQLServer, Oracle), Programming
languages (e.g., Python, Pearl, visual Basics, C++), Visualizations programs (e.g., Tableau, SAS Visual, Qlikview, Spotfire), Other (audit firm tools like PACE
(EY), Halo (PwC), KAAP (KPMG), Spotlight (Deloitte) or ‘firm tool’, and external tools like IDEA and AIDA, or not specified).
3
Number of times ADA tools were used in the specific sub-category on the selected engagement in the 109 engagements. Number of tools are higher than
number of ADA use in Table 7 since several tools may be use for each phase.
Downloaded
from
by
Aasmund
Eilifsen
on
15
June
2020

57
Table 9
Continuing Engagements and New Engagements in 2016 or 2017:
Perceptions of Usefulness of ADA
Panel A: Perceived Usefulness Average Score1
Continuing
engagements2
New
engagements3
Number of engagements 95 14
In my opinion. ADA are currently useful for risk identification on this engagement 4.6 4.8
In my opinion. ADA are currently useful for substantive testing to obtain audit
evidence on this engagement.
4.6 5.2
I am personally hesitant to use ADA in this engagement with the ADA tools and
procedures currently available in our firm.
2.9 2.5
Our firm has adequate ADA tools for this engagement. 4.7 4.8
Compared to traditional audit tools and procedures, the ADA tools and procedures
currently available in our firm are improving the quality on this engagement.
4.8 5.1
Compared to traditional audit tools and procedures, the ADA tools and procedures
currently available in our firm are efficient (time. effort or cost in generating the
result) to use on this engagement.
4.3 4.8
We lack technical competence on the audit teams to use ADA tools on this
engagement.
3.1 2.7
We have sufficient specialist support (outside the core teams) for data-gathering,
handling and/or ADA when needed on this engagement.
5.0 5.6
On this engagement, the client has a positive attitude to our collection of client
data for ADA purposes.
5.0 5.2
It is easy to collect sufficient data for ADA purposes from the client on this
engagement.
3.9 4.1
Panel B: Number of times ADA was used in the specific sub-category on the
selected engagement
Total 570 196
Average number of times per engagement 6.0 14.0
Notes:
1
Average score of the auditors on the engagements by rating scale: 1-Strongly disagree, 4-Neutral, 7-Strongly agree.
2
Engagements that were taken on by the audit firm in 2015 or earlier.
3
Engagements that were new for the audit firm in 2016 or 2017.
Downloaded
from
by
Aasmund
Eilifsen
on
15
June
2020
View publication stats

Aasmund (2020).pdf

More Related Content

Similar to Aasmund (2020).pdf (20)

Recently uploaded (20)

Aasmund (2020).pdf