experiment-loop

experiment-loop is a command for coding agents from LigphiDonk/Oh-my--paper. It costs 18 tokens per session (599 once invoked), scanned A, a copy of experiment, MIT.

A guided research-experiment workflow that designs, runs, and reviews experiments one round at a time.

In plain words
What is it for?
Use it to design a plan with datasets, baselines, settings, and metrics, implement and run it, record each result, and decide whether to adjust, repeat, or stop based on the success threshold.
Why use it?
It ensures each experiment plan is reviewed before implementation and that the next decision uses recorded results.

Command

Install

Getting it into your agent

One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.

agentmods
npx agentmods add commands/ligphidonk/oh-my--paper/experiment-loop
Clone the repo
git clone --depth 1 https://github.com/LigphiDonk/Oh-my--paper

Wrote this? Show the measurements

A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.

agentmods badge for experiment-loop

README.md
[![agentmods](https://agentmods.dev/badge/commands/ligphidonk/oh-my--paper/experiment-loop.svg)](https://agentmods.dev/commands/ligphidonk/oh-my--paper/experiment-loop)
Your own site
<a href="https://agentmods.dev/commands/ligphidonk/oh-my--paper/experiment-loop"><img src="https://agentmods.dev/badge/commands/ligphidonk/oh-my--paper/experiment-loop.svg" alt="Measured on agentmods" height="20"></a>
Per session 18 Only the description is in the session, so the agent can decide to use it. The body loads when it is invoked.
When invoked 599 The whole file, excluding the scripts and references it only reads on demand.
Security scan A 0 findings. Scan, not verified.
Origin 86% copy Near-identical to another mod in the catalogue.
Token cost

What it costs to keep this loaded

Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.

ModelPer sessionOnce invoked
Fable 5 $0.00018 $0.00599
Opus 5 $0.00009 $0.00300
Sonnet 5 $0.00004 $0.00120
Haiku 4.5 $0.00002 $0.00060

Measured 5d ago against content hash 7e01e5fdd57b, method: parsed. Prices are Anthropic first-party input rates as of 2026-08-30, from the pricing page.

Security

Grade A, and why

experiment-loop scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 5d ago.

A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.

Nothing flagged

None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.

Origin

This is a copy

86% identical to experiment — 4 lines differ, which has more behind it and is treated as the original. This page carries a canonical link to it rather than competing with it.

templates/harness/commands/experiment-loop.md · 76 lines

What it actually says

必须使用 AskUserQuestion 工具进行所有确认步骤,不得用纯文字替代。

你是 Oh My Paper Orchestrator。实验不能盲目启动,每轮都需要确认。

第一步:读取当前状态

cat .pipeline/memory/project_truth.md
cat .pipeline/memory/experiment_ledger.md
cat .pipeline/docs/research_brief.json

AskUserQuestion 展示当前实验背景:

选定方向:[project_truth 中的创新点] 已有实验:[experiment_ledger 条数,或"尚无"] 成功标准:[successThreshold]

准备进入实验循环。第一步是设计实验方案。

选项:

  • 继续,先设计方案
  • 我先描述一下我想要的实验配置
  • 取消

如果用户有自己的配置描述,先记录下来再进入设计。

第二步:设计实验方案

/codex:rescue 阅读 .pipeline/memory/project_truth.md 和 .pipeline/memory/experiment_ledger.md(避免重复失败配置),使用 .claude/skills/inno-experiment-dev/SKILL.md 设计实验方案,写入 .pipeline/docs/experiment_plan.md,不要写代码

读取 experiment_plan.md,用 AskUserQuestion 展示方案摘要,等确认:

实验方案

  • 数据集:...
  • 基线:...
  • 超参:...
  • 评估指标:...

确认后开始实现和运行。

选项:

  • 方案可以,开始实现
  • 调整某个配置
  • 重新设计方案

第三步:实现并运行

/codex:rescue --background --resume 根据 .pipeline/docs/experiment_plan.md 实现实验代码到 experiments/ 目录并运行,将每次运行结果追加到 .pipeline/memory/experiment_ledger.md

第四步:结果回来后,由你决定下一步

读取 experiment_ledger.md 最新行,向用户展示结果,用 AskUserQuestion 询问:

最新实验结果:[指标] = [值] 成功标准:[threshold] 状态:达标 ✅ / 未达标 ❌

选项(未达标时):

  • 调整超参,再跑一轮
  • 修改实验设计,重新来
  • 这个方向有问题,返回 /idea-forge
  • 结果够用了,进入写作

选项(达标时):

  • 很好,进入 /paper-sprint
  • 还想多跑几组对比实验
Changes

What this file has done since we first saw it

Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.

  1. 5d ago First seen · 76 lines · 18 tokens per session scan A 7e01e5fdd57b

Subscribe to this mod's changes

experiment-loop is a command published in the GitHub repository LigphiDonk/Oh-my--paper (721 stars, last pushed 4mo ago), licensed MIT. It adds 18 tokens to every session and 599 once invoked, about $0.0001 per session on Opus 5. A static security scan graded it A with 0 findings. It is 86% identical to experiment, differing in 4 lines, and is treated as a copy.