Getting it into your agent
One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.
npx skills add JayRHa/AgentSkills --skill ab-test-analyzergit clone --depth 1 https://github.com/JayRHa/AgentSkillsWrote this? Show the measurements
A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.
[](https://agentmods.dev/skills/jayrha/agentskills/ab-test-analyzer)<a href="https://agentmods.dev/skills/jayrha/agentskills/ab-test-analyzer"><img src="https://agentmods.dev/badge/skills/jayrha/agentskills/ab-test-analyzer/github.svg" alt="Measured on agentmods" height="20"></a>Or the 80×15 button, for a site that already has a row of RSS and ATOM ones. Only the verdict fits; the numbers stay here.
<a href="https://agentmods.dev/skills/jayrha/agentskills/ab-test-analyzer"><img src="https://agentmods.dev/badge/skills/jayrha/agentskills/ab-test-analyzer.svg" alt="Reviewed on agentmods" width="80" height="20"></a>What it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5.1 | $0.00135 | $0.02045 |
| Opus 5.5 | $0.00054 | $0.00818 |
| Sonnet 5.5 | $0.00027 | $0.00409 |
| Haiku 4.5 | $0.00014 | $0.00204 |
Grade A, and why
ab-test-analyzer scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 1mo ago.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Nothing flagged
None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.
How it starts
The opening of the file, as written. The whole thing — 105 lines — stays where its author put it; the contents beside it link to each section on GitHub.
A/B Test Analyzer
Overview
This skill helps design and analyze controlled online experiments (A/B and A/B/n tests) with statistical rigor and honest interpretation. It covers the full lifecycle: hypothesis framing, sample-size and duration planning, choosing the right test, computing p-values and confidence intervals, and avoiding the traps that produce false wins.
Keywords: A/B test, split test, experiment, hypothesis, sample size, power, MDE, minimum detectable effect, statistical significance, p-value, confidence interval, conversion rate, lift, two-proportion z-test, t-test, chi-square, peeking, multiple comparisons, Simpson's paradox, SRM, novelty effect.
Use this skill whenever someone wants to plan an experiment, decide if a result is real, or sanity-check an analysis someone else did.
Core Mental Model
- An A/B test estimates a causal effect by randomizing units (usually users) into control and treatment.
- You are testing a null hypothesis (no difference) against an alternative (there is a difference). A p-value is
P(data this extreme or more | null is true)— NOT the probability the null is true, and NOT the probability your variant is better. - Two error types: Type I (false positive, rate = alpha, typically 0.05) and Type II (false negative, rate = beta; power = 1 - beta, typically 0.80).
- You must fix sample size and test duration BEFORE you start. Stopping when significant ("peeking") inflates the false-positive rate dramatically.
Workflow
Follow these steps in order. Do not skip planning steps even when only asked to "analyze results" — verify the plan was sound first.
-
Frame the hypothesis. Turn the vague goal into a falsifiable statement: "Changing X will increase metric M from baseline b by at least the MDE, because [mechanism]." Identify ONE primary metric (the Overall Evaluation Criterion / OEC). Pre-register guardrail metrics. See
references/methodology.mdfor OEC selection. -
Pick the metric type.
- Binary / rate (conversion, click, signup) → two-proportion test.
- Continuous (revenue per user, time on page, order value) → Welch's t-test (means).
- More than 2 variants → chi-square (rates) or ANOVA-style + correction.
What ships with it
6 files beside SKILL.md in the same directory: the scripts, references and assets a skill reads on demand. Not counted in the per-session cost; read them before you install if any of them is executable.
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- 1mo ago First seen · 105 lines · 135 tokens per session scan A 3f52fcd8ef91
ab-test-analyzer is a skill published in the GitHub repository JayRHa/AgentSkills (12 stars, last pushed 2mo ago), licensed MIT. It adds 135 tokens to every session and 2,045 once invoked, about $0.0005 per session on Opus 5.5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-31.
Other skills, from other repositories
cost-optimizer
Trigger when the user asks to audit Claude Code costs, reduce token spend, says "my Claude bill is too high", "optimize my CLAUDE.md", "why is this project burning tokens", or "/cost-optimizer". Scans a project for the common Claude Code cost leaks and returns a prioritized fix list.
excalidraw-architecture
Trigger when the user asks for an architecture diagram, says "draw the system", "update the architecture diagram", "give me an excalidraw of this codebase", or "/excalidraw-architecture". Generates or updates an Excalidraw JSON file at docs/architecture.excalidraw by reading the codebase's key entry points.
model-cost-compare
Trigger when the user asks which model to use, wants to compare model costs, says "what's cheapest for this task", "should I use Opus or Sonnet", "can a smaller model handle this", or "/model-cost-compare". Estimates token cost across Opus 4.6, Sonnet 4.6, GLM-5.1, Minimax M2.7, and local Gemma 4, then recommends the…
openclaw-debugger
Trigger when an OpenClaw agent is broken, silent, crashing, stuck, not responding, returning empty output, or the user says "my agent is down", "agent not working", "/openclaw-debugger". Walks through the standard OpenClaw 2026.4 diagnosis checklist and prints a report.
git-commit-writer
Trigger when the user asks to write a commit message, generate a commit from staged changes, review a diff before committing, or says "/commit". Reads git diff --staged and produces an opinionated, convention-matching commit message.
growth-ideas
Describe your product or project and get three actionable growth ideas tailored to your stage.