Getting it into your agent
One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.
npx skills add shaan-ad/pm-os --skill experiment-reviewgit clone --depth 1 https://github.com/shaan-ad/pm-osWrote this? Show the measurements
A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.
[](https://agentmods.dev/skills/shaan-ad/pm-os/experiment-review)<a href="https://agentmods.dev/skills/shaan-ad/pm-os/experiment-review"><img src="https://agentmods.dev/badge/skills/shaan-ad/pm-os/experiment-review/github.svg" alt="Measured on agentmods" height="20"></a>Or the 80×15 button, for a site that already has a row of RSS and ATOM ones. Only the verdict fits; the numbers stay here.
<a href="https://agentmods.dev/skills/shaan-ad/pm-os/experiment-review"><img src="https://agentmods.dev/badge/skills/shaan-ad/pm-os/experiment-review.svg" alt="Reviewed on agentmods" width="80" height="20"></a>What it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5.1 | $0.00031 | $0.01553 |
| Opus 5 | $0.00015 | $0.00776 |
| Sonnet 5 | $0.00006 | $0.00311 |
| Haiku 4.5 | $0.00003 | $0.00155 |
Grade A, and why
experiment-review scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 10d ago.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Nothing flagged
None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.
How it starts
The opening of the file, as written. The whole thing — 193 lines — stays where its author put it; the contents beside it link to each section on GitHub.
Experiment Review and A/B Test Analysis
You are a PM with strong analytical skills reviewing an experiment. Your job is to assess results with statistical rigor, avoid common pitfalls (peeking, underpowered tests, Simpson's paradox), and produce a clear recommendation backed by evidence.
Consult references/stat-sig-guide.md for statistical methodology when performing calculations.
Initialization
- Read
knowledge/pm-context.mdfor product context and success metrics. - Read files in
knowledge/experiments/for past experiment results and learnings. - Read
knowledge/metrics/for baseline metric values.
Gather Experiment Details
Ask these questions in sequence. Do not skip any.
Step 1: Hypothesis
Ask:
What was the hypothesis for this experiment? State it in the format: "If we [change], then [metric] will [direction] because [reason]."
If the user does not have a formal hypothesis, help them articulate one from their description.
Step 2: Experiment Design
Ask:
- What were the variants? (control vs. treatment, or multiple treatments)
- What was the randomization unit? (user, session, device)
- What was the primary metric? Any secondary metrics?
- What was the minimum detectable effect (MDE) you designed for?
- How long has the experiment been running?
- What was the target sample size?
Step 3: Results
Ask:
- What are the conversion rates (or metric values) for each variant?
- What is the sample size per variant?
- Do you have confidence intervals or p-values already calculated?
- Were there any issues during the experiment? (logging bugs, traffic allocation errors, external events)
If the user provides a URL to results (e.g., an analytics dashboard), use WebFetch to retrieve the data.
Statistical Analysis
Step 1: Power and Sample Size Check
Before analyzing results:
- Calculate whether the experiment had sufficient sample size for the stated MDE
- If underpowered, flag this prominently: the results may not be reliable
- Calculate the actual MDE detectable with the given sample size
What ships with it
1 file beside SKILL.md in the same directory: the scripts, references and assets a skill reads on demand. Not counted in the per-session cost; read them before you install if any of them is executable.
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- 10d ago First seen · 193 lines · 31 tokens per session scan A 37255793a013
experiment-review is a skill published in the GitHub repository shaan-ad/pm-os (31 stars, last pushed 5mo ago), licensed MIT. It adds 31 tokens to every session and 1,553 once invoked, about $0.0002 per session on Opus 5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-30.
Other skills, from other repositories
spike-consumer-adversarial
OI-3 spike harness — heavy consumer, ADVERSARIAL arm. Worst-case early-exit test: the mid-workflow Skill call has no continuation guardrail and the guidance skill ends with a final-sounding anchor. Use only when explicitly invoked by the spike harness with a TRIALID and data path.
testing-laravel
Writes Laravel tests using PHPUnit. Use when "write tests", "add tests", "phpunit", "laravel test", "feature test", "unit test", "mock", "factory", or testing controllers, models, services, actions, jobs, artisan commands, or API endpoints.
testing-react
Writes React/TypeScript tests using Vitest and React Testing Library. Use when "write react tests", "vitest", "component test", "hook test", "RTL", "testing library", "snapshot test", or testing React components, hooks, and utilities.
ia-orchestrating-swarms
Coordinate multi-agent swarms for parallel and pipeline workflows. Use when coordinating multiple agents, running parallel reviews, building pipeline workflows, or implementing divide-and-conquer patterns with subagents.
ia-verification-before-completion
Enforces fresh verification evidence before any completion claim. Use when about to claim "tests pass", "bug fixed", "done", "ready to merge", handing off work, or before editing when a request has ambiguous scope.
ia-writing-tests
Generic test writing discipline: test quality, real assertions, anti-patterns, and rationalization resistance. Use when writing tests, adding test coverage, or fixing failing tests for any language or framework. Complements language-specific skills.