Getting it into your agent
One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.
npx skills add ericrisco/rsc-harness --skill ab-testinggit clone --depth 1 https://github.com/ericrisco/rsc-harnessWrote this? Show the measurements
A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.
[](https://agentmods.dev/skills/ericrisco/rsc-harness/ab-testing)<a href="https://agentmods.dev/skills/ericrisco/rsc-harness/ab-testing"><img src="https://agentmods.dev/badge/skills/ericrisco/rsc-harness/ab-testing/github.svg" alt="Measured on agentmods" height="20"></a>Or the 80×15 button, for a site that already has a row of RSS and ATOM ones. Only the verdict fits; the numbers stay here.
<a href="https://agentmods.dev/skills/ericrisco/rsc-harness/ab-testing"><img src="https://agentmods.dev/badge/skills/ericrisco/rsc-harness/ab-testing.svg" alt="Reviewed on agentmods" width="80" height="20"></a>- NVIDIA SkillSpector warn
SkillSpector: 1 finding, up to medium
These are SkillSpector’s own severities. On a checked sample its high-severity flags on skills were ~96% false positives — a documented command, a public API, a “never do X” rule — so we show them as a caution to read, not a verdict. Why →
- medium analysis-evasion · line 1 Suspicious Unicode normalization or mixed-script contentFix: Review the flagged content for security risks. Ensure no credentials, secrets, or sensitive data are exposed.
What it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5.1 | $0.00079 | $0.02450 |
| Opus 5.5 | $0.00032 | $0.00980 |
| Sonnet 5.5 | $0.00016 | $0.00490 |
| Haiku 4.5 | $0.00008 | $0.00245 |
Grade A, and why
ab-testing scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 1mo ago.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Nothing flagged
None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.
How it starts
The opening of the file, as written. The whole thing — 160 lines — stays where its author put it; the contents beside it link to each section on GitHub.
A/B testing — design and read a defensible experiment
An experiment without a pre-committed sample size and a single primary metric is not an experiment. It is a dashboard you stare at until it tells you what you wanted to hear. The discipline lives almost entirely before traffic ships: a falsifiable hypothesis, one primary metric, a sample size derived from the smallest effect worth detecting, and a stop rule you cannot renegotiate at 2pm on day four.
Pre-test checklist — every line true before any traffic
Each one is a place experiments die silently.
- A falsifiable hypothesis — names the change, the direction, and the metric it moves.
- Exactly ONE primary metric. More than one primary = multiple comparisons = inflated false positives.
- Guardrail metrics — what you refuse to harm (latency, refunds, unsubscribes) even for a win.
- The randomization unit = the analysis unit (usually the user). Mixing them is pseudoreplication.
- An MDE — the smallest lift that would change a decision. Not "any difference."
- A computed sample size and the duration it implies at your real daily eligible traffic.
- A fixed stop rule — a date or an n you commit to before launch. No "we'll see how it looks."
Step 1 — Hypothesis and metrics
State a null you can reject. "The new checkout button changes purchase conversion" with H0: conversion equal across arms, H1: it differs. Vague aspirations ("improve the funnel") have no rejection region.
Pick one primary metric and freeze it. Why: every extra primary metric is another coin flip at α, so three "primary" metrics turn a 5% false-positive rate into roughly 14%. Demote the rest to secondary.
Randomize on the same unit you analyze on. If a user sees the variant on every visit, randomize by user, not by session — analyzing 50k sessions from 8k users treats correlated observations as independent and fabricates significance.
Bad: "We think the redesign will improve engagement and revenue and retention." (no null, 3 primaries, no number)
Good: "H0: 30-day purchase conversion is equal between control and the new one-click button.
H1: it differs. Primary: purchase conversion. Guardrails: refund rate, p95 checkout latency.
Randomize by user_id. MDE: +1.5pp absolute on a 12% baseline."
What ships with it
5 files beside SKILL.md in the same directory: the scripts, references and assets a skill reads on demand. Not counted in the per-session cost; read them before you install if any of them is executable.
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- 1mo ago First seen · 160 lines · 79 tokens per session scan A 9e283bf5c950
ab-testing is a skill published in the GitHub repository ericrisco/rsc-harness (227 stars, last pushed today), licensed MIT. It adds 79 tokens to every session and 2,450 once invoked, about $0.0003 per session on Opus 5.5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-30.
Other skills, from other repositories
merge-aliases
Folds two surface names for the same backend system into one canonical entity, keeping every original mention individually retrievable, and refuses to merge pairs that only share spelling.
anchor-and-lock
Consults a check that sits outside the loop system before finalizing any decision the frozen facts bear on, and refuses every attempt by a loop to rewrite a node marked frozen, regardless of how convergent the loop's own reasoning looks.
trim-to-budget
Trims a task-scoped candidate subgraph down to a hard node-count budget, keeping the anchor plus the highest-relevance nodes, dropping any edge that touches a dropped node, and recording the exact cutoff score used.
scan-contradictions
Scans the fact graph for pairs of edges that cannot both hold true at once and flags each such pair, running as its own standing pass rather than as part of whatever task happens to be reading the graph.
counter-metric-check
Compares a watched loop's headline reading against an independently owned counter-metric reading for each period in a governance graph, and produces a governance edge for every period where the counter-metric crosses its recorded ceiling.
extract-facts
Extracts Service/Incident/Cause facts from an incident-report document against a fixed schema, rejecting anything the schema doesn't define.