Why Sampling-Based Audit Is an Endangered Practice
Internal audit has operated on a statistical sampling model for decades: test a representative subset, extrapolate conclusions to the population, and accept that a sampling approach will miss some things. This approach was necessary when testing was manual and populations were large. It is no longer necessary — and the shift away from sampling toward full-population AI testing is one of the most significant changes in audit methodology in a generation.
The implications extend beyond efficiency. Sampling-based audits have systematic blind spots: they find what is present in the sample, and sophisticated irregularities can be structured to avoid sample selection. AI full-population testing eliminates those blind spots. Every transaction, every control application, every exception — examined, not estimated.
For Canadian financial institutions, government agencies, and regulated entities, this shift is redefining what internal audit can assure and at what cost.
Continuous Control Monitoring: Audit Without the Annual Cycle
Continuous control monitoring (CCM) is the operational foundation of AI-enhanced internal audit. Instead of testing controls at audit time — a point-in-time assessment of whether controls were working when auditors looked — CCM tests controls against live data on an ongoing basis, generating real-time visibility into control performance.
A CCM deployment for a financial services client might monitor: segregation of duties violations in the ERP (flagging cases where approval and entry functions are performed by the same user), payment authorisation compliance (identifying payments above threshold that were not dual-approved), account reconciliation completion (alerting when reconciliations are late or unreconciled items exceed tolerance), and user access anomalies (detecting access pattern deviations that may indicate credential compromise).
Each of these controls is tested against the full transaction population, continuously. Exception items are queued for management response and tracked to resolution. The audit function's role shifts from periodic testing to exception review and systemic pattern analysis — higher-value work with more coverage.
Remolda's decision support analytics include CCM implementation for Canadian financial services and government clients.
Anomaly-Based Risk Flagging
Beyond rule-based control testing, AI anomaly detection identifies patterns that don't fit established norms — even when no specific rule has been violated. This is the AI capability most relevant to detecting novel fraud, operational errors that fall between defined controls, and emerging risks that rule sets have not yet captured.
Anomaly detection models learn what normal looks like for a given entity, process, or account — the distribution of transaction sizes, frequencies, counterparties, and timing that characterises legitimate activity. Deviations from this baseline trigger flags for human review, without requiring that the specific deviation be pre-defined as a rule.
For procurement fraud detection — a persistent challenge for Canadian government organisations and large enterprises — anomaly detection identifies vendor patterns, employee approval patterns, and invoice characteristics that differ from established norms. It does not replace the judgment required to assess whether an anomaly represents fraud, error, or a legitimate exception, but it surfaces items that rule-based testing would not catch.
OSFI and Treasury Board Compliance Context
Canadian financial institutions face specific regulatory obligations that internal audit AI must address. OSFI's Guideline E-23 on model risk management applies to AI models, including those used in the audit function itself. Financial institutions using AI anomaly detection or risk scoring in their audit processes must document model methodology, validate model outputs, and assess model risk — the same discipline required for credit and market risk models.
Treasury Board of Canada Secretariat policy for federal departments and Crown corporations includes requirements for internal controls over financial reporting (ICFR) that are increasingly being met through CCM implementations. The 2024 TBS Directive on Internal Audit provides a framework within which AI-enhanced audit methodology fits — though explicit guidance on AI use in audit is still developing.
For both OSFI and TBS contexts, the key audit committee communication is: AI tools expand coverage and speed of control testing, but human audit judgment remains essential for risk assessment, root cause analysis, and management remediation guidance.
See Remolda's compliance and strategy governance services for regulatory compliance implementation support.
Audit Population Analysis: Risk Scoring and Prioritisation
Before testing, internal audit functions must decide where to focus limited resources. Traditional audit plan prioritisation uses risk assessments — surveys and workshops with business units that are time-consuming to conduct and subject to availability bias (auditors find what they look for).
AI risk scoring supplements survey-based assessment with quantitative signals: operational metrics that correlate with control weakness (system change frequency, headcount turnover in control-critical roles, exception rate trends from previous audit periods), financial signals (variance patterns in account balances, transaction volume anomalies), and external signals (regulatory enforcement actions in comparable entities, industry fraud patterns from public sources).
The AI risk score is an input to audit plan prioritisation, not the decision itself. Audit judgment must assess whether quantitative signals reflect actual risk or are explained by legitimate business changes. The value is ensuring that quantitative signals are not missed when allocating scarce audit resources.
Implementing AI Audit Capabilities: A Practical Roadmap
For Canadian internal audit functions, a practical AI implementation follows a staged approach. Start with CCM on a single high-volume, rule-based process where the control parameters are clearly defined and the data is accessible — accounts payable is the most common starting point. Measure false positive rates, remediation follow-through, and examiner time per exception. Expand to additional processes as the operating model is proven.
Anomaly detection is a later-stage capability requiring more data history and more sophisticated model development — typically phase 2 or 3 in an AI audit roadmap.
Full AI risk scoring for audit plan prioritisation is a mature capability that benefits from having CCM data (which provides quantitative control performance history) as an input — making it naturally a phase 3 development.
The audit committee and senior management communication throughout this journey matters as much as the technical implementation. Boards increasingly expect internal audit to use data analytics; the communication task is helping them understand what AI audit can and cannot assure, and why coverage has improved while sample-based limitations have been reduced.
Remolda's analytics and agent implementations for internal audit are designed for the Canadian regulatory context. Contact us to discuss your audit automation roadmap.