Assurance and Audit

There are broader and narrower definitions of assurance. But all reflect some kind of three-party arrangement; see Assurance is Based on a Three-Party Model, illustrating concepts from ISACA 2013b and IAASB 2013.

Assurance is Based on a Three-Party Model is one common scenario:

This is a simplified view of what can be a more complex process and set of relationships. The ISAE 3000 [ISAE 2013] states that there must be at least three parties to any assurance engagement:

But there may be additional parties:

ISAE 3000 goes on to stipulate a complex set of business rules for the allowable relationships between these parties. Perhaps the most important rule is that the practitioner cannot be the same as either the responsible party or the intended users. There must be some level of professional objectivity.

What is the difference between assurance and simple consulting? There are two major factors:

Finally, the concept of assurance criteria is key. Some assurance is executed against the responsible party’s own criteria. In this form of assurance, the primary questions are “are you documenting what you do, and doing what you document?”. That is, for example, “do you have formal process management documentation? And are you following it?”.

Other forms of assurance use external criteria. A good example is the Uptime Institute’s data center tier certification criteria, discussed below. If criteria are weak or non-existent, the assurance engagement may be more correctly termed an advisory effort. Assurance requires clarity on this topic.

The general topic of “assurance” implies a spectrum of activities. In the strictest definitions, assurance is provided by licensed professionals under highly formalized arrangements. However, while all audit is assurance, not all assurance is audit. As noted in COBIT for Assurance: “Assurance also covers evaluation activities not governed by internal and/or external audit standards” [ISACA 2013a).

This is a blurry boundary in practice, as an assurance engagement may be undertaken by auditors, and then might be casually called an “audit” by the parties involved. And there is a spectrum of organizational activities that seem at least to be related to formal assurance:

Some of these activities may be managed primarily internally, but even in the case of internally managed activities, there is usually some sense of governance, some desire for objectivity.

From a purist perspective, internally directed assurance is a contradiction in terms. There is a conflict of interest in that in terms of the three-party model above, the accountable party is the practitioner.

However, it may well be less expensive for an organization to fund and sustain internal assurance capabilities and get much of the same benefits as from external parties. This requires sufficient organizational safeguards to be instituted. Internal auditors typically report directly to the Board-level Audit Committee and, generally, are not seen as having a conflict of interest.

In another example, an internal compliance function might report to the corporate general counsel (chief lawyer), and not to any executive whose performance is judged based on their organization’s compliance – this would be a conflict of interest. However, because the internal compliance function is ultimately under the CEO, their concerns can be overruled.

The various ways that internal and external assurance arrangements can work, and can go wrong, is a long history. If you are interested in the topic, review the histories of Enron Corporation, Worldcom, the 2008 mortgage crisis, and other such failures.

Risk management (discussed in the previous section) may be seen as part of a broader assurance ecosystem. For evidence of this, consider that the Institute of Internal Auditors offers a certificate in Risk Management Assurance; The Open Group also operates the Open FAIR™ Certification Program. Assurance in practice may seem to be biased towards risk management, but (as with governance in general) assurance as a whole relates to all aspects of IT and digital governance, including effectiveness and efficiency.

Audit practices may be informed of known risks and particularly concerned with their mitigation, but risk management remains a distinct practice. Audits may have scope beyond risks, and audits are only one tool used by risk management; see Assurance and Risk Management.

In short, and as shown in the above diagram, assurance plays a role across value recognition, while risk management specifically targets the value recognition objective of risk optimization.

Before we turn to a more detailed discussion of the audit, we will discuss some specifically non-audit examples of assurance seen in IT and digital management.

Example 1: Due Diligence on a Cloud Provider

Your company is considering a major move to cloud infrastructure for its systems. The agility value proposition – the ability to minimize cost of delay – is compelling, and there may be some cost structure advantages as well.

But you are aware of some cloud failures:

The question is, how do you manage the risks of trusting your data and organizational operations to a cloud provider? This is not a new question, as computing has been outsourced to specialist firms for many years. You want to be sure that their operations meet certain standards such as:

Data center evaluations of cloud providers are a form of assurance. Two well-known approaches are:

The Uptime Institute provides the well-known “Tier” concept for certifying data centers, from Tier I to Tier IV. In their words: “certification provides assurances that there are not shortfalls or weak links anywhere in the data center infrastructure” [Uptime 2016]. The Tiers progress as follows [Uptime 2014]:

Uptime Institute certification is a generic form of assurance in terms of the three-party model; the data center operator must work with the Uptime Institute who provides an independent opinion based on their criteria as to the data center’s tier (and therefore effectiveness).

The SOC 3 report is considered an “assurance” standard as well. However, as mentioned above, this is the kind of “assurance” done in general by licensed auditors, and which might casually be called an “audit” by the participants. A qualified professional, again in the three-party model, examines the data center in terms of the AICPA SSAE-18 standard.

Your internal risk management organization might look to both Uptime Institute and SOC 3 certification as indicators that your cloud provider risk is mitigated. (More on this in Risk and Compliance Management.)

Example 2: Internal Process Assessment

You may also have concerns about your internal operations. Perhaps your process for selecting technology vendors is unsatisfactory in general; it takes too long and yet vendors with critical weaknesses have been selected. More generally, the actual practices of various areas in your organization may be assessed by external consultants using the related guidance:

These assessments may be performed through using a maturity scale; e.g., CMM-derived. The CMM-influenced ISO/IEC 15504-5:2012 may be used as a general process assessment framework. (Remember that we have discussed the problems with the fundamental CMM assumptions on which such assessments are based.)

According to Bente et al.: “In our own experience, we have seen that the maturity models have their limitations” [Bente et al. 2012]. They warn that maturity assessments of Enterprise Architecture at least are prone to being:

Those issues may well apply to all forms of maturity assessments. Let the buyer beware. At least, the concept of maturity should be very carefully defined in a manner relevant to the organization being assessed.

Example 3: Competitive Benchmarking

Finally, you may wonder: “How does my digital operation compare to other companies?” Now, it is difficult to go to a competitor and ask this. It is also not especially practical to go and find some non-competing company in a different industry you do not understand well. An entire industry has emerged to assist with this question.

We talked about the role of industry analysts in Investment and Portfolio. Benchmarking firms play a similar role and, in fact, some analyst firms provide benchmarking services.

There are a variety of ways benchmarking is conducted, but it is similar to assurance in that it often follows the three-party model. Some stakeholder directs an accountable party to be benchmarked within some defined scope. For example, the number of staff required to managed a given quantity of servers (aka admin:server) has been a popular benchmark. (Note that with cloud, virtualization, and containers, the usefulness of this metric is increasingly in question.)

An independent authority is retained. The benchmarker collects, or has collected, information on similar operations; for example, they may have collected data from 50 organizations of similar size on admin:server ratios. This data is aggregated and/or anonymized so that competitive concerns are reduced. Wells Fargo^® will not be told: “JP Morgan Chase™ has an overall ratio of 1:300”; they will be told: “The average for financial services is 1:250”.

In terms of formal assurance principles, the benchmark data becomes the assessment criteria. A single engagement might consider dozens of different metrics, and where simple quantitative ratios do not apply, the benchmarker may have a continuously maintained library of case studies for more qualitative analysis. This starts to shade into the kind of work also performed by industry analysts. As the work becomes more qualitative, it also becomes more advisory and less about “assurance” per se.

There are two major kinds of auditors of interest to us:

An external auditor can be defined as someone who is “chartered by a regulatory authority to visit an enterprise or entity and to review and independently report the results of that review.” [Moeller 2013].

Many accounting firms offer external audit services, and the largest accounting firms (such as PriceWaterhouse Coopers and EY) provide audit services to the largest organizations (corporations, non-profits, and governmental entities). External auditors are usually certified public accountants, licensed by their state, and following industry standards (e.g., from the American Institute of Certified Public Accountants).

By contrast, internal auditing is housed internally to the organization, as defined by the Institute of Internal Auditors: “Internal auditing is an independent appraisal function established within an organization to examine and evaluate its activities as a service to the organization” [Moeller 2013].

Internal audit is considered a distinct but complementary function to external audit [Cadbury 1992]. The internal audit function usually reports to the Audit Committee. As with assurance in general, independence is critical – auditors must have organizational distance from those they are auditing, and must not be restricted in any way that could limit the effectiveness of their findings.

As with other forms of assurance, audit follows the three-party model. There is a stakeholder, an accountable party, and an independent practitioner. The typical internal audit lifecycle consists of (derived from ISACA 2013a):

In the scoping phase, the parties are identified (e.g., the Board Audit Committee, the accountable and responsible parties, the auditors, and other stakeholders).

The scope of the audit is very specifically established, including objectives, controls, and various governance elements (e.g., processes) to be tested. Appropriate frameworks may be utilized as a basis for the audit, and/or the organization’s own process documentation.

The audit is then performed. A variety of techniques may be used by the auditors:

A fundamental principle is “expected versus actual”. There must be some expected result to a process step, calculation, etc., to which the actual result can be compared.

Finally, the audit results are reported to the agreed users (often with a preliminary “heads up” cycle so that people are not surprised by the results). Deficiencies are identified in various ways and typically are taken into system and process improvement projects.

Evidence of Notability

Assurance as a broad topic, and audit as a narrower instance, like governance itself are fundamental to the functioning of the economy. ISACA has published significant guidance on both topics, as have other, broader organizations such as the International Auditing and Assurance Standards Board (IAASB).

Limitations

Assurance and audit in general require some clear standard or expectation against which to assess. In terms of process, they are suitable for lower-variability problem domains. They are less applicable to complex or chaotic domains. However, even within a chaotic situation, there will still be basic assumptions around, for example, how money and security are to be handled. Audit, in the IT systems context, has been important enough to drive the creation of one of the major IT/Digital Practitioner organizations, the ISACA (founded in 1967 as the EDPAA).

Related Topics