exp

Experiment Management

Standardized experiment workflow: design → run → record → compare → collect for paper.

Sub-commands

$ARGUMENTS[0] determines the action. If omitted, show help.

---

/swf:exp new <title>

Create a new experiment record from template.

1. Ask experiment type:

   • benchmark — Phase 2 标准化对比实验（和 baseline 公平对比）
   • case — Phase 3 Case Study / Feature Demo（深入分析，讲故事）

2. Auto-number: scan the corresponding subdirectory for next number

   • Benchmark → docs/experiments/benchmark/bench-<NNN>-<slug>.md
   • Case → docs/experiments/case/case-<NNN>-<slug>.md

3. Ask type-specific questions:

   Benchmark:

   • 和哪些 baseline 比较？（方法名、来源、版本）
   • 用哪些数据集？
   • 评估哪些指标？
   • 公平性控制：相同数据划分？重复次数？超参调优方式？

   Case Study:

   • 为什么选这个 case？生物学背景？
   • 这个 case 要展示什么能力？
   • 数据来源（GEO ID 等）？
   • 论文里这个 case 想讲什么故事？

4. Generate record from corresponding template in ${CLAUDE_SKILL_DIR}/../reference/experiment-template.md

5. Auto-capture environment:

   python --version
   pip list | grep -E "torch|scanpy|numpy|scipy|pandas"
   hostname
   nvidia-smi --query-gpu=name,memory.total --format=csv,noheader 2>/dev/null
   

6. Generate config file if applicable → configs/<prefix>-<NNN>.yaml

7. Generate run command and write into record

8. Commit the record and config

9. Create results directory: mkdir -p results/<prefix>-<NNN>

10. Remind: "实验记录已创建。运行实验后，用 /swf:exp log <prefix>-<NNN> 记录结果。"

---

/swf:exp log <NNN>

Record results for a completed experiment.

1. Read the experiment record docs/experiments/exp-<NNN>-*.md
2. Collect results:
   • Check results/exp-<NNN>/ for output files
   • If metrics file exists (.json, .csv), auto-extract key metrics
   • If figures exist, list them with paths
   • If log file exists, extract final metrics (loss, accuracy, etc.)
3. Ask the user:
   • 结果是否符合预期？
   • 关键发现是什么？
   • 有什么后续需要做？
4. Update the experiment record:
   • Fill in results table with metrics
   • Fill in figure references
   • Fill in conclusion
   • Update status to Completed (or Failed)
   • Record the current git commit hash
5. Commit the updated record
6. Prompt next steps:
   • "要和其他实验比较吗？(/swf:exp compare)"
   • "要更新 design.md 吗？(/swf:update-design)"
   • If results invalidate assumptions: "实验结果和 design.md 假设不一致，建议写 ADR 记录 (/swf:adr)"