agent-governance

Agent Governance

Add explicit policy, trust, and audit controls around agent behavior before "helpful" automation becomes unbounded automation.

When to Use

• Designing an agent that can call tools, APIs, databases, or shell commands
• Reviewing a multi-agent workflow that needs trust boundaries between specialists
• Defining when an action should be allowed, denied, or escalated for approval
• Adding audit requirements, rate limits, or policy configuration to an existing system

When NOT to Use

Instead of agent-governance	Use
Broad repository security scoring	evaluate-repository
OWASP ASI checklist review	agent-owasp-check
Architecture-first threat modeling	threat-model-analyst

Governance Model

Think in this order:

User request -> intent classification -> policy check -> tool execution -> audit log
                  ↓                       ↓                ↓
             threat signals         allow/review/deny   trust update

A useful governance layer usually needs five parts:

1. policy definition
2. intent classification
3. tool-level enforcement
4. trust scoring
5. append-only audit trail

Workflow

1. Define policy as configuration

Start with a serializable policy instead of hardcoding checks across business logic.

Include:

• allowed_tools
• blocked_tools
• blocked_patterns
• max_calls_per_request
• require_human_approval
• profile
• tool_policies

Example shape:

name: production-agent
allowed_tools:
  - search_documents
  - query_database
blocked_tools:
  - shell_exec
  - delete_record
blocked_patterns:
  - "(?i)(api[_-]?key|secret|password)\\s*[:=]"
max_calls_per_request: 25
profile: balanced
require_human_approval:
  - send_email
tool_policies:
  shell_exec:
    rate_limit: 3/hour
    approval: required
    justification: ticket-or-incident
  query_database:
    rate_limit: 30/request
    approval: not-required