experiment-management

Experiment Management

This skill is activated whenever the user talks about experiments. Read .claude/rules/experiments.md for the full rules. This skill handles the infrastructure — folder structure, dashboard sync, state tracking.

Coordination with Other Design Skills

When the user is designing an experiment, another design/brainstorming skill may also be active (e.g., superpowers:brainstorming). If so, the two are not in conflict — they handle different concerns:

• The design skill drives the conversation: clarifying questions, proposing approaches, presenting design, getting approval.
• This skill drives the infrastructure: folder creation, dashboard sync, state tracking, lifecycle gates.

When a design skill is active alongside this one:

1. The design skill runs the conversation — questions, approaches, design presentation.
2. The experiment folder is still created immediately by this skill — don't wait for the design process to finish.
3. The design spec goes to EXPERIMENT_README.md in the experiment folder, NOT the design skill's default spec location (e.g., docs/superpowers/specs/). This skill's output location takes precedence for experiments.
4. After the user approves the design, the next step is the experiment lifecycle (update flow_state.json → /raca:experiment-preflight → canary → run), NOT the design skill's default next step (e.g., writing-plans). If implementation planning is needed for experiment code, it can be used within the lifecycle, but the experiment flow owns the top-level sequence.

When no design skill is active: This skill handles the full design conversation itself, following the flow defined in .claude/rules/experiments.md.

Experiment Folder Structure

Every experiment lives at notes/experiments/<experiment-name>/. Create this structure immediately when a conversation turns to a concrete experiment — do not wait for the design to be "complete":

notes/experiments/<name>/
├── experiment.yaml          # Config: hypothesis, models, tasks, conditions
├── EXPERIMENT_README.md     # What this experiment is about, results, conclusions
├── HUGGINGFACE_REPOS.md     # Table of all HF datasets produced (newest first)
├── questions.md             # Research questions (READ ONLY — never edit)
├── red_team_brief.md        # What could go wrong, validation criteria
├── flow_state.json          # Current phase + state (machine-readable)
├── activity_log.jsonl       # Timeline of events (append-only)
├── user/                    # User's personal notes (RACA doesn't touch these)
│   ├── README.md
│   ├── FINDINGS.md
│   ├── DECISIONS.md
│   └── summary.md
└── experiments/             # Sub-experiments (focused follow-ups)
    └── <sub-name>.md