agentic_coding_lab: Skill for Claude Code

Skill: run-rq

End-to-end orchestration for advancing a single research question (RQ) in this lab repo. Pure orchestration — every operation calls existing repo scripts; no new Python or Bash code is written.

Argument

• RQ-N (e.g. RQ-model-quality) or a direct path to an RQ dir.
• If not given: infer from the last user turn, otherwise ask back ("Which RQ? e.g. RQ-model-quality").

Repo conventions (from the top-level README.md and memory)

• RQ dirs live in four subtrees: research/questions-claude/<chapter>-*/ (Claude-Code RQs), research/questions-opencode/<chapter>-*/ (OpenCode RQs), research/questions-cross/<chapter>-*/ (cross-harness RQs), and research/workflow-dev/<chapter>-*/ (workflow evolution). The <chapter> prefix (e.g. 2.6) is an ordering label, not an id — the stable identity is the frontmatter id: (e.g. RQ-lean). Each RQ dir holds README.md, findings.md, runs.csv, summary.md.
• Resolving an RQ-<slug> id to a path (the dir name carries a chapter number, not the id): grep all subtrees for the frontmatter id:. Anchor with ^id: and a trailing boundary so the whole slug must match exactly (no slug is a prefix of another, so an exact-line match is unambiguous):

  RQ_DIR=$(grep -rlE "^id:[[:space:]]*RQ-model-quality[[:space:]]*$" \
             research/questions-claude/*/README.md \
             research/questions-opencode/*/README.md \
             research/questions-cross/*/README.md \
             research/workflow-dev/*/README.md \
           2>/dev/null | head -1 | xargs -r dirname)
  

  On no match → ask the user. On multiple → take the first and inform the user. Pass "$RQ_DIR" to all scripts below (they accept any path and write outputs to the dir).
• Mandatory frontmatter fields: id, question, factors, controls, outcomes, min_replicates. There is no status field — whether the RQ needs work is read off the data (experiments/rq-status.py, rules in experiments/rq_facts.py). Only an RQ that ends without full data gets closed: "<reason>", and only on the user's decision.
• Methodology constraint: baseline-oneshot-* / baseline-iterative-* only with prompt: prose; every other arm with all three styles. If factors.workflow_x_prompt exists, no additional factors.workflow / controls.workflow is allowed.
• Active katas: claim-office, game-of-life, sphinx-score, game-of-life-cli, claim-office-lite, mars-rover. controls.kata_base must be from this set. Each has the three prompt variants (-prose, -user-story, -example-mapping); all but mars-rover also have a <basename>-verification/ suite, so verification_pct is available there.
  • The list is not a ranking, but the pool is lopsided in practice: claim-office and game-of-life carry the bulk of the runs, sphinx-score is the established small quality kata, and mars-rover is near-unused. Prefer a kata that already has runs in neighbouring RQs — a fill on a fresh kata has no reference cells to compare against.
  • Check the actual kata dir before rejecting an RQ on this list. The list is hand-maintained and has lagged behind the repo before (sphinx-score was in use in three RQs while still missing here). ls experiments/katas/ is the authority; this line is a convenience copy.
• Model IDs are lab-variant IDs (opus-4-7, opus-4-7-no-thinking, opus-4-6-portkey, opus-4-6-portkey-no-thinking, sonnet-4-6, sonnet-4-6-no-thinking, sonnet-4-6-portkey, sonnet-4-6-portkey-no-thinking, haiku-4-5, haiku-4-5-no-thinking, haiku-4-5-portkey, haiku-4-5-portkey-no-thinking). The -portkey suffix marks models routed via the Portkey gateway.
• Aggregation is query-based: ALL runs in experiments/runs/ matching the selector query count — regardless of which batch produced them.
• Batch plan is idempotent: counts existing matches and only fills missing replicates up to min_replicates. Only ok runs count as replicates, so a timeout or aborted run is planned again.