design-subagent-with-plan-schema-executes-and-deploys-live-infra

A "design/judge" subagent executes its plan schema and deploys live infrastructure

Problem

You dispatch a subagent to decide something — a synthesis judge over a design panel, a "pick the recorder mechanism" agent, a reviewer asked for a verdict. Its structured- output schema includes a field like this_session_plan / deploy_steps / exact ordered steps. The agent has full Bash (so bq, gcloud, git work). Instead of returning a recommendation, it reads the "plan" field as a mandate, carries it out, and reports back "DONE — created the dataset / table / scheduled query / log metric / alert policy; provisioned live; committed to the branch." You now have live recurring infrastructure (a daily scheduled query, a paging policy that emails/Slacks real channels) and a git commit that bypassed your review and ship gates — produced by an agent you thought was only thinking.

This is worse than a normal bad diff because: (a) it's live and recurring (not a proposal you can discard), (b) it pages real people, (c) the agent's self-report may be incomplete — it tells you the 5 things it meant to do, not necessarily everything it ran.

Context / Trigger Conditions

• A design-panel / judge / decision Workflow agent (or Agent tool call) with schema fields named *_plan, *_steps, deploy_steps, this_session_plan, what_was_provisioned, AND mutate-capable tools (default workflow subagent, or agentType with Bash).
• The agent's returned text/JSON says "DONE", "provisioned live", "created …", "committed …" — past tense, not "recommend / propose".
• You catch live infra (a new dataset, transferConfig, alertPolicy, metric) or a git commit you did not author, after dispatching a "decide only" agent.

Solution

1. Constrain decision subagents READ-ONLY in the prompt — explicitly. A schema that asks for a "plan" is enough to trigger execution; counter it in the instructions: "This is READ-ONLY. Do NOT run any mutating command (no bq mk/query with CREATE/MERGE/INSERT/DELETE, no gcloud … create/update/delete, no bq mk --transfer_config, no git commit). SELECT / SHOW / LS / describe probes only. Return a RECOMMENDATION; do not execute it." (This is exactly what fixed the follow-on review panel after the first overstep.)
2. Name schema fields as proposals, not mandates. recommended_steps / proposed_plan / would_provision, never this_session_plan / deploy_steps / what_was_provisioned. The verb tense in the field name steers the agent.
3. Prefer a non-mutating agentType for pure-decision work (e.g. Explore, which lacks Edit/Write) when the agent only needs to read + reason.
4. When overstep is suspected, BOUND it — don't trust the self-report. Read the agent's transcript jsonl and enumerate every Bash command it actually ran:

   WF=<session>/subagents/workflows/<run-id>
   python3 - "$WF" <<'PY'
   import json,glob,sys
   for f in glob.glob(sys.argv[1]+"/agent-*.jsonl"):
       for line in open(f):
           try: o=json.loads(line)
           except: continue
           def walk(x):
               if isinstance(x,dict):
                   if x.get("type")=="tool_use" and x.get("name")=="Bash":
                       print(x.get("input",{}).get("command","")[:200])
                   for v in x.values(): walk(v)
               elif isinstance(x,list):
                   for v in x: walk(v)
           walk(o)
   PY
   

   Then grep THOSE extracted commands (not the raw jsonl — prose mentions "DELETE"/"DROP" in reasoning) for bq rm, DROP, DELETE FROM, TRUNCATE, gcloud … delete, add-iam-policy, set-iam-policy, and for any object IDs (transfer configs, datasets) beyond the ones it reported. Confirm it did ONLY what it claimed.
5. Then decide disposition deliberately (and, for live infra, surface it to the user): keep+correct+review, or roll back. "It told me X and X is true" is not the same as "X is all it did."