Getting it into your agent
One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.
npx agentmods add skills/ericrisco/rsc-harness/ab-testingnpx skills add ericrisco/rsc-harness --skill ab-testinggit clone --depth 1 https://github.com/ericrisco/rsc-harnessWrote this? Show the measurements
A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.
[](https://agentmods.dev/skills/ericrisco/rsc-harness/ab-testing)<a href="https://agentmods.dev/skills/ericrisco/rsc-harness/ab-testing"><img src="https://agentmods.dev/badge/skills/ericrisco/rsc-harness/ab-testing.svg" alt="Measured on agentmods" height="20"></a>What it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5 | $0.00079 | $0.02450 |
| Opus 5 | $0.00039 | $0.01225 |
| Sonnet 5 | $0.00016 | $0.00490 |
| Haiku 4.5 | $0.00008 | $0.00245 |
Grade A, and why
ab-testing scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 4d ago.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Nothing flagged
None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.
How it starts
The opening of the file, as written. The whole thing — 160 lines — stays where its author put it; the contents beside it link to each section on GitHub.
A/B testing — design and read a defensible experiment
An experiment without a pre-committed sample size and a single primary metric is not an experiment. It is a dashboard you stare at until it tells you what you wanted to hear. The discipline lives almost entirely before traffic ships: a falsifiable hypothesis, one primary metric, a sample size derived from the smallest effect worth detecting, and a stop rule you cannot renegotiate at 2pm on day four.
Pre-test checklist — every line true before any traffic
Each one is a place experiments die silently.
- A falsifiable hypothesis — names the change, the direction, and the metric it moves.
- Exactly ONE primary metric. More than one primary = multiple comparisons = inflated false positives.
- Guardrail metrics — what you refuse to harm (latency, refunds, unsubscribes) even for a win.
- The randomization unit = the analysis unit (usually the user). Mixing them is pseudoreplication.
- An MDE — the smallest lift that would change a decision. Not "any difference."
- A computed sample size and the duration it implies at your real daily eligible traffic.
- A fixed stop rule — a date or an n you commit to before launch. No "we'll see how it looks."
Step 1 — Hypothesis and metrics
State a null you can reject. "The new checkout button changes purchase conversion" with H0: conversion equal across arms, H1: it differs. Vague aspirations ("improve the funnel") have no rejection region.
Pick one primary metric and freeze it. Why: every extra primary metric is another coin flip at α, so three "primary" metrics turn a 5% false-positive rate into roughly 14%. Demote the rest to secondary.
Randomize on the same unit you analyze on. If a user sees the variant on every visit, randomize by user, not by session — analyzing 50k sessions from 8k users treats correlated observations as independent and fabricates significance.
Bad: "We think the redesign will improve engagement and revenue and retention." (no null, 3 primaries, no number)
Good: "H0: 30-day purchase conversion is equal between control and the new one-click button.
H1: it differs. Primary: purchase conversion. Guardrails: refund rate, p95 checkout latency.
Randomize by user_id. MDE: +1.5pp absolute on a 12% baseline."
What ships with it
5 files beside SKILL.md in the same directory: the scripts, references and assets a skill reads on demand. Not counted in the per-session cost; read them before you install if any of them is executable.
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- 4d ago First seen · 160 lines · 79 tokens per session scan A 9e283bf5c950
ab-testing is a skill published in the GitHub repository ericrisco/rsc-harness (60 stars, last pushed yesterday), licensed MIT. It adds 79 tokens to every session and 2,450 once invoked, about $0.0004 per session on Opus 5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-30.
Other skills, from other repositories
ab-testing-ecommerce
Run controlled experiments on product pages, checkout flows, and pricing to find what converts best using statistical significance testing.
A/B Test Designer
Design statistically valid A/B tests for marketing optimization.
featurevisor
Author, query, and integrate Featurevisor — Git-based feature flags, A/B experiments, and remote config. Use whenever the user mentions Featurevisor, works in a project containing featurevisor.config.js, edits files under attributes/, segments/, features/, variables/, groups/, schemas/, targets/, sets/, or tests/…
A/B Test Validation
Validating A/B test implementations including traffic splitting accuracy, statistical significance calculation, metric tracking, and experiment cleanup.
research-collaborator
Use this skill whenever a researcher wants to test, validate, stress-test, or falsify a research idea or hypothesis — especially in AI/ML/deep learning. Trigger on phrases like "I have an idea," "would this work," "test this hypothesis," "sanity check my idea," "what's wrong with this idea," "review my results," "is…
conversion-optimizer
!cat skills/shared/protocols/ux-protocol.md 2>/dev/null || true !cat skills/shared/protocols/input-validation.md 2>/dev/null || true !cat skills/shared/protocols/tool-efficiency.md 2>/dev/null || true !cat .production-grade.yaml 2>/dev/null || echo "No config — using defaults".