wicked-garden-agentic-trust-and-safety

Trust and Safety

Essential patterns for building safe, trustworthy, production-ready agentic systems.

Core Principles

1. Least Privilege: Agents have minimum necessary permissions
2. Defense in Depth: Multiple layers of safety checks
3. Fail Safe: Errors should fail toward safety, not capability
4. Human Oversight: High-stakes decisions require human approval
5. Auditability: All decisions and actions are logged and traceable
6. Graceful Degradation: System remains safe even when components fail

Human-in-the-Loop Gates

When to Add Human Gates

Always require approval for:

• Production data modifications (delete, update critical data)
• Financial transactions above threshold
• Communications to external parties
• Credential or security changes
• Irreversible operations

Consider approval for:

• First-time operations
• Operations outside normal patterns
• Low-confidence decisions
• Operations near resource limits

Implementation Pattern

async def execute_with_approval(action, threshold=0.8):
    if action.confidence < threshold or action.is_high_stakes():
        approval = await request_human_approval(action)
        if not approval.approved:
            raise ApprovalDenied(approval.reason)
    return await action.execute()

Approval Workflow Design

Synchronous Approval: Block until human responds (for urgent decisions) Asynchronous Approval: Queue for later review (for batch operations) Escalation Chains: Route to higher authority if primary approver unavailable Timeout Handling: Define what happens if no approval received

See refs/guardrails-input-output.md, refs/guardrails-actions.md, and refs/guardrails-resources.md for detailed implementation patterns.

Output Validation

Structured Output Validation

Force outputs into validated schemas using Pydantic or similar.

Content Validation

Check outputs before acting on them:

• Format validation: Matches expected structure
• Range validation: Numeric values within acceptable bounds
• Completeness validation: Required fields are present
• Consistency validation: Outputs are internally consistent