Audit Documentation & Governance
Master the end-to-end AI audit process: planning, evidence collection, findings classification, report writing, and remediation tracking. Covers audit documentation standards, governance structures, and practical audit checklists.
6.1 — The AI Audit Lifecycle
An AI audit follows a structured lifecycle from initial planning through remediation follow-up. Understanding each phase and its deliverables is critical for effective auditing.
Planning phase: Define audit objectives, scope (which AI systems, which controls), criteria (against which framework — NIST, ISO 42001, EU AI Act), timeline, and resource requirements. Engage stakeholders early — AI audits require cooperation from data science, engineering, legal, and business teams.
Systems affecting individuals' rights, safety, or financial outcomes receive the deepest audit scrutiny.
Automated decision-making without human oversight requires more extensive testing and control evaluation.
Large user base, sensitive data, or public-facing applications increase the scope requirements.
Systems subject to specific regulations (EU AI Act high-risk, RBI guidelines) need compliance-focused scoping.
Auditors must NOT have development responsibilities for the system being audited. An effective audit team includes: technical auditors (ML expertise), governance/compliance auditors, domain experts, and legal advisors. Independence is a fundamental audit principle — expect exam questions on this.
6.2 — Evidence Collection
Evidence collection is the foundation of any audit finding. Without properly collected, documented, and preserved evidence, findings lack credibility and cannot withstand challenge.
If an organization cannot provide model cards, risk assessments, or impact assessments, this absence is itself an audit finding. Document what was requested, when, and what was not provided. Missing documentation often indicates deeper governance gaps.
Evidence must be documented with: source, date collected, collection method, relevance to audit criteria, and chain of custody. Digital evidence should be timestamped and stored securely. This mirrors financial audit evidence standards.
6.3 — Findings Classification and Reporting
Findings classification ensures that the most critical issues receive immediate attention while providing a structured framework for remediation planning.
| Severity | Description | Required Action Timeline |
|---|---|---|
| Critical | Immediate risk to individuals or regulatory non-compliance | Immediate action required |
| High | Significant control weakness or material gap | Action within 30 days |
| Medium | Control improvement needed, moderate risk | Action within 90 days |
| Low | Best practice recommendation, advisory | No mandatory timeline |
What was expected — the standard, requirement, or control that should be in place.
What was actually found — the factual observation during the audit.
Why the gap exists — root cause analysis of the deficiency.
What is the risk or impact — the potential harm if not addressed.
What should be done — specific, actionable remediation steps.
Criteria: ISO 42001 requires AI impact assessments before deployment. Condition: The credit scoring model was deployed without an impact assessment. Cause: No formal pre-deployment review process exists. Consequence: Potential unfair treatment of loan applicants; regulatory non-compliance. Recommendation: Implement mandatory pre-deployment impact assessment gate with documented approval.
The audit report must be accessible to non-technical executives. Structure: Executive summary, scope/methodology, system description, findings by severity, management response (accept/partially accept/reject with action plan, responsible party, and target date), and appendices with detailed test results.
6.4 — AI Governance Structures
Effective AI governance requires clear accountability structures, from board-level oversight to operational teams. The three lines of defense model provides a proven framework.
| Artifact | Purpose | Review Frequency |
|---|---|---|
| AI Policy | Sets organizational principles and boundaries for AI use | Annually |
| AI Risk Appetite Statement | Defines acceptable risk levels for AI systems | Annually |
| AI System Register/Inventory | Catalogs all AI systems with risk classifications | Continuously updated |
| Model Risk Management Framework | Governs model development, validation, and monitoring | Annually |
| Data Governance Framework | Ensures data quality, provenance, and privacy compliance | Annually |
| Incident Response Plan | Procedures for AI-specific failures and incidents | Annually + after incidents |
| Responsible AI Principles | Ethical guidelines for AI development and deployment | Biennially |
Shadow AI — unauthorized use of AI tools by employees (e.g., uploading confidential data to ChatGPT) — is an emerging governance challenge. Governance must address the full AI supply chain: in-house models, fine-tuned models, third-party APIs, open-source components, training data, and shadow AI.
Show Answer
Criteria (what was expected), Condition (what was found), Cause (why the gap exists), Consequence (what is the risk/impact), and Recommendation (what should be done).
Show Answer
First line: AI development and operations teams own and manage risks. Second line: AI risk and compliance function oversees and challenges. Third line: Internal audit provides independent assurance.
Show Answer
Documentation review (model cards, risk assessments, etc.), technical testing (independent evaluation, fairness testing, red-teaming), interviews (structured interviews with key personnel), and process observation (verifying procedures match practice).
Show Answer
Critical (immediate risk, immediate action required), High (significant weakness, action within 30 days), Medium (improvement needed, 90 days), Low (best practice recommendation, advisory).
Show Answer
AI Policy, AI Risk Appetite Statement, AI System Register/Inventory, Model Risk Management Framework, Data Governance Framework, Incident Response Plan, and Responsible AI Principles. These should be reviewed and updated annually.
Show Answer
Auditors must not have development responsibilities for the system being audited. Independence ensures objective evaluation free from conflicts of interest. An auditor who developed the system cannot objectively assess their own work. This is a fundamental professional audit principle.
Show Answer
Shadow AI is unauthorized use of AI tools by employees (e.g., uploading confidential data to public AI services). It's a governance concern because it creates data leakage risks, compliance violations, and uncontrolled AI usage outside the organization's risk management framework.
Show Answer
Structure: Executive summary, scope/methodology, system description, findings by severity, management response, and appendices. The primary audience is non-technical executives and board members, so reports must translate technical findings into business risk language.