ai-workflow

/dotclaude:ai-workflow — LLM workflow + cost discipline kit

You are setting up the AI-workflow safety layer — the discipline that prevents night-long eval runs from producing four-figure surprise bills, and that catches the mock-mode-bypass class of bug at the source rather than in the invoice.

The eval-cost-watcher agent is the showpiece. It reads the diff, counts fixtures, projects cost ranges, and suggests cheaper alternatives — all BEFORE the eval runs.

Phase 1 — Read the project's AI shape

Before any question:

1. Detect AI SDK presence — does the project actually have LLM calls?

   grep -rln "@anthropic-ai/sdk\|openai\|@ai-sdk\|@google/generative-ai\|gemini\|claude" \
     package.json Cargo.toml pyproject.toml requirements.txt 2>/dev/null | head
   grep -rn "from anthropic\|from openai\|import.*@anthropic" \
     --include="*.py" --include="*.ts" --include="*.js" . 2>/dev/null | head -10
   

   If NO AI dependencies, STOP. Report applicability check failed. Tell the user this skill doesn't apply.

2. AI workflow directories — find where prompts / configs / fixtures live:

   find . -type d \( -name "ai" -o -name "llm" -o -name "prompts" -o -name "evals" \
     -o -name "workflows" -o -name "agents" -o -name "fixtures" \) \
     -not -path "*/node_modules/*" 2>/dev/null | head
   find . -path ./node_modules -prune -o \
     \( -name "*.prompt.*" -o -name "*system-prompt*" -o -name "*instruction*" \) \
     -print 2>/dev/null | head
   

3. Eval suite — find the eval entry points:

   cat package.json 2>/dev/null | grep -A 2 '"scripts"' | grep -E "eval|test:ai|test:llm"
   find . -name "*.eval.*" -o -name "*eval*.py" -o -name "*eval*.ts" \
     -not -path "*/node_modules/*" 2>/dev/null | head
   ls evals/ tests/eval/ tests/llm/ 2>/dev/null
   

4. Model identifiers — read the config:

   grep -rEn "MODEL_NAME|model.*=.*\"|model: ['\"]|MODEL_ID|model_id" \
     --include="*.ts" --include="*.py" --include="*.json" --include="*.yaml" . \
     2>/dev/null | head -30
   

   Note which model tiers the project uses (frontier-large vs frontier-mid vs frontier-fast). Each tier swap is a 5-10× cost multiplier.