Scope
This guide tackles the challenge of ensuring your AI agents perform precisely as instructed, without deviation. If your team deploys AI systems that act on behalf of users, like booking resources or modifying configurations, you need a framework to measure intent alignment to prevent compliance incidents.
This guide covers:
- Defining and measuring the gap between user intent and AI action
- Implementing frameworks to monitor AI agent behavior
- Mapping intent-alignment controls to compliance requirements
- Building audit trails that capture both requests and interpretations
This guide does not cover model training, prompt engineering, or AI ethics.
Key Concepts
Intent Gap: The difference between what a user requests and the actions an AI system takes. AI researcher Simon Willison noted that Anthropic's Fable AI was "relentlessly proactive," indicating it interpreted requests broadly and acted beyond explicit instructions.
Genie Coefficient: A metric to quantify the intent gap. A coefficient of zero means the AI did exactly what was asked. Higher values indicate additional actions or assumptions beyond the user's request.
AI Harness: The control layer between user input and AI execution. It includes input validation, action allowlists, rollback mechanisms, and approval workflows. Your harness design determines the potential size of an intent gap.
Underspecified Intent: When a user request lacks detail for deterministic execution. For example, "Book me a flight to Chicago" is underspecified without details like airport, date, or budget. AI systems fill these gaps, which can lead to misalignment.
Requirements Breakdown
Your AI agent's behavior intersects with several compliance domains:
Access Control (PCI DSS v4.0.1 Requirement 7.2.2)
If your AI accesses cardholder data, it must use unique IDs and authenticate each access. An AI that "proactively" queries customer records can create an audit gap if the user didn't authorize that access.
Change Management (ISO/IEC 27001:2022 Control 8.32)
AI agents modifying configurations or deploying code must follow your change control process. If an AI interprets "fix the authentication bug" as "rewrite the entire auth module," it results in an unauthorized change.
Audit Logging (SOC 2 CC6.2)
Log both what the AI was asked to do and what it actually did. A single log entry like "AI completed user request" isn't enough. You need the original user input, AI's interpretation, actions taken, and any assumptions made.
Least Privilege (NIST 800-53 Rev 5 AC-6)
Your AI should have the minimum permissions needed to fulfill user requests. Granting broad permissions because the AI "might need them" violates least privilege and increases risk.
Implementation Guidance
Step 1: Define Your Action Taxonomy
Create a comprehensive list of actions your AI can take, not just capabilities. Each action should have a risk score and required approval level.
Step 2: Instrument Request Interpretation
Log the AI's interpretation before it executes any action:
User request: "Get me the latest sales numbers"
AI interpretation:
- Action: Query sales_summary table
- Assumptions: "Latest" = current quarter-to-date
- Additional context: Includes all regions unless filtered
- Estimated scope: ~50,000 records
This log entry is your baseline for measuring the Genie coefficient.
Step 3: Build Action Comparison Logic
After execution, compare the planned actions to the actual actions:
Planned actions: [query_sales_summary]
Actual actions: [query_sales_summary, query_customer_details, send_email_summary]
Intent gap: 2 unplanned actions
Genie coefficient: 2.0 (for this request)
Track this coefficient per request, user, and AI agent version.
Step 4: Set Acceptable Thresholds
Define acceptable intent gaps with your business stakeholders:
- Read-only data queries: Coefficient ≤ 0.5
- Configuration changes: Coefficient = 0
- External communications: Coefficient = 0
- Financial transactions: Coefficient = 0
Step 5: Implement Circuit Breakers
If your AI's interpretation exceeds your threshold, halt execution and request human approval. Don't just log the gap, stop the action.
Common Pitfalls
Pitfall: Treating AI actions as user actions in audit logs
Ensure your logs distinguish between direct user actions and AI-mediated actions.
Pitfall: Granting admin permissions "for flexibility"
Constrain the AI's permissions to your action taxonomy. Expand only when new actions are explicitly approved.
Pitfall: Measuring accuracy instead of alignment
The Genie coefficient measures whether the AI stayed within bounds, not just if it completed tasks accurately.
Pitfall: Ignoring the harness in your threat model
Threat model the harness as carefully as any privileged service. It's where most operational risk lies.
Pitfall: No rollback mechanism
Ensure you can undo unauthorized AI actions. Log enough context for rollback, not just for compliance.
Quick Reference Table
| Control Domain | Requirement | Intent Alignment Control | Implementation |
|---|---|---|---|
| Access Control | PCI DSS v4.0.1 Req 7.2.2 | Unique ID per AI-mediated access | Log user ID + AI agent ID + interpreted action |
| Change Management | ISO/IEC 27001:2022 Control 8.32 | Pre-execution approval for changes | Halt AI execution when action=modify AND coefficient>0 |
| Audit Logging | SOC 2 CC6.2 | Separate logs for request vs. execution | Three log entries: user input, AI interpretation, actual actions |
| Least Privilege | NIST 800-53 Rev 5 AC-6 | Permission scoping per action type | Grant permissions per action taxonomy, not per AI agent |
| Incident Response | ISO/IEC 27001:2022 Control 5.26 | Rollback capability for AI actions | Store pre-action state + rollback procedure per action type |
Next step: Audit one AI agent deployment this week. Log 50 user requests, capture the AI's interpretation, and measure the actual actions taken. Calculate your baseline Genie coefficient before building controls around it. You can't manage what you don't measure, and you can't comply with controls you haven't instrumented.
