ab-test-designer

ab-test-designer is a skill for Claude Code from pmprompt/claude-plugin-product-management. It costs 28 tokens per session (498 once invoked), scanned A, original, MIT.

A guide for designing A/B tests, where different users see different versions of a feature or experience so the results can be compared.

In plain words
What is it for?
Use it to define hypotheses, success metrics, sample sizes, test duration, user segments, and control and variant designs.
Why use it?
It helps teams test changes with clear measures and an analysis plan instead of relying on opinion.

Skill for Claude Code

Written for Claude Code: argument-hint in frontmatter.

Part of the pmprompt plugin — 28 skills, 9 commands shipped together

Good fit Use it to define hypotheses, success metrics, sample sizes, test duration, user segments, and control and variant designs.

Compare 6 skills from other repositories ↓
Install with agentmods
npx agentmods add skills/pmprompt/claude-plugin-product-management/ab-test-designer
Install

Getting it into your agent

One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.

Any agent
npx skills add pmprompt/claude-plugin-product-management --skill ab-test-designer
Clone the repo
git clone --depth 1 https://github.com/pmprompt/claude-plugin-product-management

Made for: Claude Code.

Or install pmprompt, the plugin that ships this one along with the rest of its 28 skills, 9 commands.

Wrote this? Show the measurements

A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.

agentmods badge for ab-test-designer

README.md
[![agentmods](https://agentmods.dev/badge/skills/pmprompt/claude-plugin-product-management/ab-test-designer/github.svg)](https://agentmods.dev/skills/pmprompt/claude-plugin-product-management/ab-test-designer)
Your own site
<a href="https://agentmods.dev/skills/pmprompt/claude-plugin-product-management/ab-test-designer"><img src="https://agentmods.dev/badge/skills/pmprompt/claude-plugin-product-management/ab-test-designer/github.svg" alt="Measured on agentmods" height="20"></a>

Or the 80×15 button, for a site that already has a row of RSS and ATOM ones. Only the verdict fits; the numbers stay here.

agentmods 80×15 button for ab-test-designer

Your own site · 80×15
<a href="https://agentmods.dev/skills/pmprompt/claude-plugin-product-management/ab-test-designer"><img src="https://agentmods.dev/badge/skills/pmprompt/claude-plugin-product-management/ab-test-designer.svg" alt="Reviewed on agentmods" width="80" height="20"></a>
Per session 28 Skills are progressive disclosure: only the name and description are preloaded; the body loads when the skill is used.
When invoked 498 The whole file, excluding the scripts and references it only reads on demand.
Security scan A 0 findings. A grade says what 26 rules found in the file — not that it is safe. Third-party audits
  • Socket pass 18 Apr 2026
  • Snyk pass 18 Apr 2026
How audits are shown
Origin original No closer match found in the catalogue.
Token cost

What it costs to keep this loaded

Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.

ModelPer sessionOnce invoked
Fable 5.1 $0.00028 $0.00498
Opus 5 $0.00014 $0.00249
Sonnet 5 $0.00006 $0.00100
Haiku 4.5 $0.00003 $0.00050

Measured 10d ago against content hash bb5d8e7727b5, method: parsed. Prices are Anthropic first-party input rates as of 2026-09-10, from the pricing page.

Security

Grade A, and why

ab-test-designer scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 10d ago.

A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.

Nothing flagged

None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.

Origin

Copies of this mod

1 near-identical copy found in the catalogue:

skills/ab-test-designer/SKILL.md · 75 lines

How it starts

The opening of the file, as written. The whole thing — 75 lines — stays where its author put it; the contents beside it link to each section on GitHub.

Domain Context

This skill implements a proven product management framework. The approach combines best practices from industry leaders and is designed for practical application in day-to-day PM work.

Input Requirements

  • Context about your product, feature, or problem
  • Relevant data, research, or constraints (recommended but optional)
  • Clear articulation of what you're trying to achieve

A/B Test Designer

When to Use

  • Testing a new feature or design variation
  • Validating a hypothesis before full rollout
  • Optimizing conversion rates or key metrics
  • Choosing between multiple design approaches
  • Need to make a data-driven decision on a change

What This Skill Does

Helps you design rigorous A/B tests with clear hypotheses, success metrics, sample size calculations, and analysis plans.

Instructions

Help me design an A/B test for [feature/change]. Include:

  1. Hypothesis

    • Current situation and metrics
    • Proposed change
    • Expected impact and why
  2. Test Design

    • Primary success metric
    • Secondary metrics
    • Sample size needed
    • Test duration
    • User segments to include/exclude
  3. Variants

    • Control (A): current experience
    • Variant (B): new experience
    • Any additional variants (C, D, etc.)
  4. Risks and Controls

    • Potential negative impacts
    • Guardrail metrics
    • When to stop the test early
  5. Analysis Plan

    • Statistical significance threshold
    • How to handle edge cases
    • Decision criteria

Feature context: [Add context about the change you want to test]

Best Practices

  • Start with a clear, falsifiable hypothesis
  • Choose one primary metric to avoid multiple comparison issues
  • Calculate sample size upfront based on expected effect size
  • Run tests for full weekly cycles to account for day-of-week effects
  • Set a minimum test duration (usually 1-2 weeks)
  • Define success criteria before running the test
  • Monitor guardrail metrics (revenue, errors, performance)

Example

Input: Testing new onboarding flow vs current 3-step process Output: Hypothesis (new 1-step flow will increase co...

Read the full file on GitHub · 75 lines

Changes

What this file has done since we first saw it

Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.

  1. 10d ago First seen · 75 lines · 28 tokens per session scan A bb5d8e7727b5

Subscribe to this mod's changes

ab-test-designer is a skill published in the GitHub repository pmprompt/claude-plugin-product-management (49 stars, last pushed 6mo ago), licensed MIT. It adds 28 tokens to every session and 498 once invoked, about $0.0001 per session on Opus 5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-30.

Related

Other skills, from other repositories

utility-pm-skill-builder

Guides contributors from a PM skill idea to a complete Skill Implementation Packet aligned with pm-skills conventions. Runs gap analysis, validates through a Why Gate, classifies by type and phase, generates draft files, and writes to a staging area for review before promotion.

product-on-purpose/pm-skills · 60 tokens

utility-update-pm-skills

Validates internet access, compares the locally installed pm-skills version against the latest public release, and updates local files with conflict-aware overwrite-or-skip options. Produces an update report listing changed files, skipped files, and new capabilities. Use when you want to bring a local pm-skills…

product-on-purpose/pm-skills · 71 tokens

discover-journey-map

Maps a customer journey across stages, touchpoints, emotional curve, pain points, and moments of truth into a markdown artifact with an optional mermaid timeline or flowchart. Use when synthesizing existing research into the shape of a customer's experience, end-to-end or for one phase. Without research signal yet…

product-on-purpose/pm-skills · 88 tokens

utility-pm-skill-iterate

Applies targeted improvements to an existing pm-skills skill based on feedback, validation reports, or convention changes. Reads current files, previews proposed changes, writes on confirmation, and suggests a version bump. Use when improving a skill after validation or feedback.

product-on-purpose/pm-skills · 58 tokens

utility-pm-skill-validate

Audits an existing pm-skills skill against structural conventions and quality criteria. Produces a structured validation report with pass/fail checks, severity-graded findings, and actionable recommendations. Use when checking whether a skill meets repo standards before shipping or after making changes.

product-on-purpose/pm-skills · 60 tokens

tool-foundation-sprint-founding-hypothesis

Day 2 end capstone move of a Foundation Sprint. Compresses the sprint's full strategic frame into a single canonical sentence (the Founding Hypothesis) plus an assumption scorecard, why-we-believe, what-could-prove-us-wrong, and recommended next validation step. Use after Magic Lenses is signed. Strict canonical…

product-on-purpose/pm-skills · 109 tokens