Visionologies
GOVERNANCE•4 min read

From Sampling to Population Testing: AI in IT Audit

Traditional IT audit lived with a structural compromise. The population was too large to test exhaustively, so auditors sampled — pull twenty-five access reviews from a year of four thousand, spot-check thirty change tickets out of twelve thousand. The sample is the source of most IT audit anxiety: a twenty-five-row sample missing the one rogue ticket is mathematically near-certain when the base rate of issues is below five percent.

AI changes the math. A modest large language model can read all twelve thousand change tickets, classify them by risk profile, and surface the forty worth a human's attention — turning sampled assurance into population-level assurance for the same audit-hour budget. The same logic applies to access reviews (anomaly detection on join-move-leave patterns), to vendor SOC2 reports (NLP extraction of control gaps and drift between years), and to log review (clustering operational signals against security signals).

What auditors will need to learn is not particularly familiar to the profession yet: prompt design, evaluation methodology, and — most critically — how to evidence why the model's output should be trusted. The next IFAC and ISACA generation of standards is going to be about evidencing AI use in audit work, not about banning it. Auditors who lean in find their work scope expanding rather than shrinking, because the answer to "what about the other 99% of the population?" is no longer automatically "out of scope."

The risk is the obvious one: an audit team that delegates judgement to a model without checking edge cases ends up with a different category of opinion-formation failure than the sampling era ever produced. Stewardship of the model is now part of the audit file.

arrow_back All insights