experiment-designer

experiment-designer is a skill for Claude Code from Dokkabei97/forged-claude-code. It costs 37 tokens per session (1,075 once invoked), scanned A, original, MIT.

A guide for designing and interpreting A/B tests and other product experiments. An A/B test compares two versions with different groups of users to measure which performs better.

In plain words
What is it for?
Use it to define hypotheses, metrics, guardrails, traffic splits, sample sizes, test duration, and statistical conclusions for product or marketing experiments.
Why use it?
It replaces opinion-led decisions with a plan for measuring a stated hypothesis, estimating the needed sample and duration, and interpreting the results.

Skill for Claude Code

Written for Claude Code: shipped in a Claude Code plugin.

Part of the forged-claude-code plugin — 29 skills, 14 commands, 9 agents, 2 hooks shipped together

Good fit Use it to define hypotheses, metrics, guardrails, traffic splits, sample sizes, test duration, and statistical conclusions for product or marketing experiments.

Compare 6 skills from other repositories ↓
Install with agentmods
npx agentmods add skills/dokkabei97/forged-claude-code/experiment-designer
Install

Getting it into your agent

One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.

Any agent
npx skills add Dokkabei97/forged-claude-code --skill experiment-designer
Clone the repo
git clone --depth 1 https://github.com/Dokkabei97/forged-claude-code

Made for: Claude Code.

Or install forged-claude-code, the plugin that ships this one along with the rest of its 29 skills, 14 commands, 9 agents, 2 hooks.

Wrote this? Show the measurements

A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.

agentmods badge for experiment-designer

README.md
[![agentmods](https://agentmods.dev/badge/skills/dokkabei97/forged-claude-code/experiment-designer.svg)](https://agentmods.dev/skills/dokkabei97/forged-claude-code/experiment-designer)
Your own site
<a href="https://agentmods.dev/skills/dokkabei97/forged-claude-code/experiment-designer"><img src="https://agentmods.dev/badge/skills/dokkabei97/forged-claude-code/experiment-designer.svg" alt="Measured on agentmods" height="20"></a>
Per session 37 Skills are progressive disclosure: only the name and description are preloaded; the body loads when the skill is used.
When invoked 1,075 The whole file, excluding the scripts and references it only reads on demand.
Security scan A 0 findings. A grade says what 26 rules found in the file — not that it is safe.
Origin original No closer match found in the catalogue.
Token cost

What it costs to keep this loaded

Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.

ModelPer sessionOnce invoked
Fable 5.1 $0.00037 $0.01075
Opus 5 $0.00018 $0.00537
Sonnet 5 $0.00007 $0.00215
Haiku 4.5 $0.00004 $0.00108

Measured 7d ago against content hash 6bcaef7ed5c8, method: parsed. Prices are Anthropic first-party input rates as of 2026-09-07, from the pricing page.

Security

Grade A, and why

experiment-designer scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 7d ago.

A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.

Nothing flagged

None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.

skills/cpo/experiment-designer/SKILL.md · 152 lines

How it starts

The opening of the file, as written. The whole thing — 152 lines — stays where its author put it; the contents beside it link to each section on GitHub.

Experiment Designer - A/B Test & Product Experiment Guide

Overview

Designs statistically sound experiments so you make decisions based on data, not opinions. Especially critical when traffic is low and every experiment counts.


1. When to Apply

Trigger Behavior
Testing a product change Full experiment design
"A/B test", "experiment" Interactive experiment builder
Interpreting test results Statistical analysis guide

2. Experiment Design Template

# Experiment: [Name]

## Hypothesis
> If we [change X], then [metric Y] will [improve/decrease] by [Z%],
> because [rationale based on user behavior/data].

## Design
| Field | Value |
|-------|-------|
| **Type** | A/B / A/B/n / Multivariate |
| **Primary Metric** | [e.g., conversion rate] |
| **Guardrail Metrics** | [metrics that should NOT worsen] |
| **Baseline** | [current value of primary metric] |
| **MDE** | [Minimum Detectable Effect, e.g., 5%] |
| **Significance** | 95% (α = 0.05) |
| **Power** | 80% (β = 0.20) |

## Sample Size & Duration
| Variant | Traffic Split | Required Sample | Est. Duration |
|---------|--------------|----------------|---------------|
| Control (A) | 50% | [N] | [days] |
| Variant (B) | 50% | [N] | [days] |

## Variants
### Control (A)
[Current experience — no changes]

### Variant (B)
[Specific change being tested]

## Success Criteria
- **Win**: Primary metric improves by ≥ MDE AND guardrails stable
- **Lose**: Primary metric does not improve OR guardrails degrade
- **Inconclusive**: Not enough data (extend or abandon)

## Risks & Mitigations
- [Risk]: [Mitigation]

3. Sample Size Quick Reference

For conversion rate experiments (95% significance, 80% power):

Baseline Rate MDE 5% MDE 10% MDE 20%
1% 380K 95K 24K
5% 72K 18K 4.6K
10% 34K 8.6K 2.2K
20% 16K 3.9K 1K
50% 3.1K 780 200

Per variant. Total = N × number of variants.

Read the full file on GitHub · 152 lines

Changes

What this file has done since we first saw it

Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.

  1. 7d ago First seen · 152 lines · 37 tokens per session scan A 6bcaef7ed5c8

Subscribe to this mod's changes

experiment-designer is a skill published in the GitHub repository Dokkabei97/forged-claude-code (2 stars, last pushed 6mo ago), licensed MIT. It adds 37 tokens to every session and 1,075 once invoked, about $0.0002 per session on Opus 5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-31.