Getting it into your agent
One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.
npx agentmods add skills/jayrha/agentskills/ab-test-analyzernpx skills add JayRHa/AgentSkills --skill ab-test-analyzergit clone --depth 1 https://github.com/JayRHa/AgentSkillsWhat it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5 | $0.00135 | $0.02045 |
| Opus 5 | $0.00068 | $0.01022 |
| Sonnet 5 | $0.00027 | $0.00409 |
| Haiku 4.5 | $0.00014 | $0.00204 |
Grade A, and why
ab-test-analyzer scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 2d ago.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Nothing flagged
None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.
How it starts
The opening of the file, as written. The whole thing — 105 lines — stays where its author put it; the contents beside it link to each section on GitHub.
A/B Test Analyzer
Overview
This skill helps design and analyze controlled online experiments (A/B and A/B/n tests) with statistical rigor and honest interpretation. It covers the full lifecycle: hypothesis framing, sample-size and duration planning, choosing the right test, computing p-values and confidence intervals, and avoiding the traps that produce false wins.
Keywords: A/B test, split test, experiment, hypothesis, sample size, power, MDE, minimum detectable effect, statistical significance, p-value, confidence interval, conversion rate, lift, two-proportion z-test, t-test, chi-square, peeking, multiple comparisons, Simpson's paradox, SRM, novelty effect.
Use this skill whenever someone wants to plan an experiment, decide if a result is real, or sanity-check an analysis someone else did.
Core Mental Model
- An A/B test estimates a causal effect by randomizing units (usually users) into control and treatment.
- You are testing a null hypothesis (no difference) against an alternative (there is a difference). A p-value is
P(data this extreme or more | null is true)— NOT the probability the null is true, and NOT the probability your variant is better. - Two error types: Type I (false positive, rate = alpha, typically 0.05) and Type II (false negative, rate = beta; power = 1 - beta, typically 0.80).
- You must fix sample size and test duration BEFORE you start. Stopping when significant ("peeking") inflates the false-positive rate dramatically.
Workflow
Follow these steps in order. Do not skip planning steps even when only asked to "analyze results" — verify the plan was sound first.
-
Frame the hypothesis. Turn the vague goal into a falsifiable statement: "Changing X will increase metric M from baseline b by at least the MDE, because [mechanism]." Identify ONE primary metric (the Overall Evaluation Criterion / OEC). Pre-register guardrail metrics. See
references/methodology.mdfor OEC selection. -
Pick the metric type.
- Binary / rate (conversion, click, signup) → two-proportion test.
- Continuous (revenue per user, time on page, order value) → Welch's t-test (means).
- More than 2 variants → chi-square (rates) or ANOVA-style + correction.
What ships with it
6 files beside SKILL.md in the same directory: the scripts, references and assets a skill reads on demand. Not counted in the per-session cost; read them before you install if any of them is executable.
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- 2d ago First seen · 105 lines · 135 tokens per session scan A 3f52fcd8ef91
ab-test-analyzer is a skill published in the GitHub repository JayRHa/AgentSkills (4 stars, last pushed 1mo ago), licensed MIT. It adds 135 tokens to every session and 2,045 once invoked, about $0.0007 per session on Opus 5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-31.
Other skills, from other repositories
reverse-engineering-tools
Guide for reverse engineering protected games and anti-cheat components across user mode, kernel mode, and hypervisor-aware environments. Use this skill when analyzing drivers, IOCTL protocols, callback registration, injected-code artifacts, integrity checks, protected binaries, or debugging security-sensitive game…
curating-bgs-modpack
Use when planning or building the whole Bethesda modpack incrementally — batch strategy, rollback point, naming convention, separator discipline, attribution, and declaring 风格 before adding mods. Triggers - "build a modpack", "plan the pack", "batch strategy", "rollback point", "naming convention", "declare 风格"…
evaluating-bgs-mods
Use when deciding whether a mod belongs in a modpack — judging mod quality, fit, risk, and pack-value BEFORE download/install. Triggers - "should I add this mod", "is this mod good", "评估这个mod", "这个mod值得装吗", "is this mod worth it", "this mod looks too good to be true", "compare these mods", "does this mod fit my pack".…
writing-modpack-changelog
Use when cutting a modpack release. Creates /docs/release-changelog.md if absent; appends a new version section with grouped changes. Triggers - 'cut a release', 'release notes', 'changelog', 'v1.2.3 changes', 'what changed since last version'.
agent-newbie-guide
Skill "agent-newbie-guide" from hashgraph-online/awesome-codex-plugins, covering codex 新手引导, 🚫 新手术语过滤（绝对禁止）, 统一行为规范（内嵌）, 📍 阶段位置 and 启动自检.
review-work
Quality gate: verify each acceptance criterion of a completed task/work unit, run quality checks, and create follow-up tasks for gaps. Use before merging or to audit delivered work. Invoked as /agiflow:review-work . Uses getworkunit, gettask, updatetask, createtask, createtaskcomment.