ablation-planner

Ablation Planner

Systematically design ablation studies that answer the questions reviewers will ask. The external LLM reviewer leads the design (reviewer perspective), CC reviews feasibility and implements.

Context: $ARGUMENTS

When to Use

• Main results pass /result-to-claim with claim_supported = yes or partial
• User explicitly requests ablation planning
• /auto-iteration-loop reviewer identifies missing ablations

Reviewer LLM Configuration (mandatory, read first)

This skill calls an external LLM reviewer. Never hardcode a model name and never read the reviewer model from task.md / project READMEs / source comments. Project-level files may list available API keys for unrelated purposes (e.g., LLM-as-judge inside experiment code); those are not the reviewer config.

Resolve LLM_MODEL, LLM_BASE_URL, LLM_API_KEY strictly in this priority order before any reviewer call:

1. Project MCP config — ${PROJECT_ROOT}/.mcp.json, field mcpServers["llm-chat"].env.{LLM_MODEL,LLM_BASE_URL,LLM_API_KEY}.
2. User MCP config — ~/.claude/settings.json, same field.
3. Shell environment — $LLM_MODEL, $LLM_BASE_URL, $LLM_API_KEY.

Pre-flight check (run before Step 2, mandatory)

LLM_MODEL_SRC=""
if [ -f .mcp.json ] && jq -e '.mcpServers["llm-chat"].env.LLM_MODEL' .mcp.json >/dev/null 2>&1 ; then
  export LLM_MODEL=$(jq -r '.mcpServers["llm-chat"].env.LLM_MODEL' .mcp.json)
  export LLM_BASE_URL=$(jq -r '.mcpServers["llm-chat"].env.LLM_BASE_URL' .mcp.json)
  export LLM_API_KEY=$(jq -r '.mcpServers["llm-chat"].env.LLM_API_KEY' .mcp.json)
  LLM_MODEL_SRC="project .mcp.json"
elif [ -f ~/.claude/settings.json ] && jq -e '.mcpServers["llm-chat"].env.LLM_MODEL' ~/.claude/settings.json >/dev/null 2>&1 ; then
  export LLM_MODEL=$(jq -r '.mcpServers["llm-chat"].env.LLM_MODEL' ~/.claude/settings.json)
  export LLM_BASE_URL=$(jq -r '.mcpServers["llm-chat"].env.LLM_BASE_URL' ~/.claude/settings.json)
  export LLM_API_KEY=$(jq -r '.mcpServers["llm-chat"].env.LLM_API_KEY' ~/.claude/settings.json)
  LLM_MODEL_SRC="user ~/.claude/settings.json"
elif [ -n "$LLM_MODEL" ] && [ -n "$LLM_BASE_URL" ] && [ -n "$LLM_API_KEY" ] ; then
  LLM_MODEL_SRC="shell env"
fi
echo "[reviewer-config] LLM_MODEL=$LLM_MODEL  LLM_BASE_URL=$LLM_BASE_URL  source=$LLM_MODEL_SRC"