agent-orchestrator-ts: Command for Claude Code

/auton — Autonomy Diagnostic

Purpose

Diagnose WHY the jleechanclaw + AO system is NOT autonomously driving PRs to N-green and merged. The system is supposed to do this without human intervention — if it isn't, something is broken.

Skill reference: .claude/skills/auton.md Session monitor skill: .claude/skills/ao-session-monitor.md

Usage

• /auton — Run full autonomy diagnostic
• /auton <description> — Focus on a specific failure mode

Execution

YOU (Claude) must execute the following steps immediately.

Step 1: Read the goals and system design (mandatory)

Before answering, read these files to understand what "working" means:

1. ~/.openclaw/CLAUDE.md — repo goals, PR green definition, autonomy target
2. ~/.codex/AGENTS.md — agent policies
3. ~/.openclaw/agent-orchestrator.yaml — AO project config
4. ~/.openclaw/SOUL.md — openclaw decision-making policy

Step 2: Run diagnostics (parallel group A + B)

Run Group A and Group B in parallel for speed.

Group A — Infrastructure & Session State

Run all of these together:

# A1. Orchestrator session — what is it doing?
tmux capture-pane -t ao-orchestrator -p -S -20 2>/dev/null || echo "ORCHESTRATOR SESSION NOT FOUND"

# A2. Worker session inventory with activity detection (30 lines each)
# Uses ao-session-monitor skill: capture 30 lines, look for Unicode activity indicators
for s in $(tmux list-sessions 2>/dev/null | grep -E "ao-[0-9]|jc-" | cut -d: -f1); do
  echo "=== SESSION: $s ==="
  last=$(tmux capture-pane -t "$s" -p -S -30 2>/dev/null)
  echo "$last" | tail -5
  echo "---"
  # Detect state via activity indicators
  activity=$(echo "$last" | grep -oE "[✻✶✳✽✾✢] [A-Za-z]+" | tail -1)
  pr=$(echo "$last" | grep -oE "#[0-9]+" | head -1)
  uc=""; echo "$last" | grep -q "uncommitted" && uc="+uncommitted"
  if [ -n "$activity" ]; then
    echo "STATE: WORKING $pr $uc ($activity)"
  elif echo "$last" | grep -qE "Running|timeout"; then
    echo "STATE: WORKING (shell command) $pr $uc"
  elif echo "$last" | grep -qE "Baked|Sautéed"; then
    echo "STATE: COMPLETED $pr"
  elif echo "$last" | grep -q "queued"; then
    echo "STATE: QUEUED $pr"
  else
    echo "STATE: IDLE $pr $uc"
  fi
  echo ""
done

# A3. Is AO lifecycle-worker running? Check per-project
lw_count=$(ps aux | grep -c "[l]ifecycle-worker")
lw_per_project=$(ps aux | grep "[l]ifecycle-worker" | awk '{print $NF}' | sort -u | wc -l | tr -d ' ')
echo "Lifecycle-worker process count: $lw_count (unique projects: $lw_per_project)"
# Only flag as duplicate if same project appears more than once
ps aux | grep "[l]ifecycle-worker" | awk '{print $NF}' | sort | uniq -d | while read dup; do
  echo "DUPLICATE lifecycle-worker for project: $dup"
  ps aux | grep "[l]ifecycle-worker" | grep "$dup"
done

# A3b. ZOMBIE SESSION DETECTION — cross-reference AO session store vs tmux
echo "--- AO session store state (agent-orchestrator project) ---"
ao session ls --project agent-orchestrator 2>/dev/null || echo "ao session ls failed"
echo "--- Zombie check: AO-marked-killed sessions still in tmux ---"
# Discover session dirs dynamically — hash varies per config path
for ns_dir in ~/.agent-orchestrator/*-agent-orchestrator/sessions; do
  [ -d "$ns_dir" ] || continue
  echo "(scanning $ns_dir)"
done
AO_DATA=$(ls ~/.agent-orchestrator/*-agent-orchestrator/sessions/ 2>/dev/null)
for s in $AO_DATA; do
  sfile=$(ls ~/.agent-orchestrator/*-agent-orchestrator/sessions/"$s" 2>/dev/null | head -1)
  [ -f "$sfile" ] || continue
  status=$(grep "^status=" "$sfile" 2>/dev/null | cut -d= -f2)
  tmux_name=$(grep "^tmuxName=" "$sfile" 2>/dev/null | cut -d= -f2)
  [ "$status" = "killed" ] || continue
  if tmux has-session -t "$tmux_name" 2>/dev/null; then
    pr=$(grep "^pr=" "$sfile" | cut -d= -f2 | grep -oE "[0-9]+$")
    echo "  ZOMBIE: $s (tmux=$tmux_name, PR=#$pr) — AO=killed but tmux still running"
  fi
done

# A4. Stray worktrees blocking spawns?
git -C "$(git rev-parse --show-toplevel 2>/dev/null || echo .)" worktree list 2>/dev/null | grep "locked" || echo "No locked worktrees"

# A5. Recent backfill claim failures (surface blocker reason, not just uncovered)
python3 - <<'PY'
import json, pathlib
log = pathlib.Path.home() / ".openclaw" / "logs" / "ao-lifecycle-agent-orchestrator.log"
if not log.exists():
    print("No lifecycle log found for claim-failure scan")
    raise SystemExit(0)
seen = {}
for raw in log.read_text(errors="ignore").splitlines():
    if "lifecycle.backfill.claim_failed" not in raw:
        continue
    try:
        payload = json.loads(raw)
    except Exception:
        continue
    if payload.get("operation") != "lifecycle.backfill.claim_failed":
        continue
    data = payload.get("data") or {}
    pr = data.get("prNumber")
    err = " ".join(str(data.get("error", "")).split())
    if isinstance(pr, int):
        seen[pr] = err
if not seen:
    print("No recent claim_failed blockers")
else:
    print("Recent claim_failed blockers:")
    for pr in sorted(seen):
        print(f"  PR #{pr}: {seen[pr][:220]}")
PY