human-oversight-design

Human Oversight Design

Autonomous AI agents that act without meaningful human control are a governance failure waiting to happen. This skill teaches you to design oversight into the system architecture — not bolt it on afterwards.

The Oversight Spectrum

Before writing any agent code, decide where your system sits:

Full Automation          Supervised Automation       Human-Led + AI-Assisted
─────────────────────────────────────────────────────────────────────────
AI decides & acts        AI decides, human approves  Human decides, AI informs
     │                          │                              │
High risk                   Medium risk                    Low risk
Requires audit trail    Requires override mechanism    Requires transparency

Rule: Systems affecting citizens, finances, health, or legal status must sit at "Supervised Automation" or further right. Argue in writing if you believe otherwise.

Step 1: Map Every Action to a Control Level

For every action your agent can take, assign a control level before implementation:

# Define your action registry with explicit oversight requirements
ACTION_REGISTRY = {
    # (action_name): (control_level, rationale)
    'read_database':          ('autonomous',  'Read-only, no external effect'),
    'generate_draft_email':   ('autonomous',  'Draft only, not sent'),
    'search_web':             ('autonomous',  'Read-only, no external effect'),
    'send_email':             ('supervised',  'External communication, irreversible'),
    'write_file':             ('supervised',  'Modifies state, potentially irreversible'),
    'delete_record':          ('supervised',  'Irreversible'),
    'submit_form':            ('supervised',  'External action with consequences'),
    'make_payment':           ('supervised',  'Financial transaction'),
    'update_citizen_record':  ('supervised',  'Affects individual entitlements'),
    'escalate_to_authority':  ('supervised',  'Triggers further action'),
    'take_legal_action':      ('blocked',     'Must not be automated'),
}

CONTROL_LEVELS = {
    'autonomous':  'Agent executes without approval',
    'supervised':  'Agent proposes, human must approve before execution',
    'blocked':     'Agent must not perform this action under any circumstances',
}