suede-ab-testing

suede-ab-testing is a skill for Claude Code, Codex from JasonColapietro/suede-creator-skills. It costs 88 tokens per session (3,367 once invoked), scanned A, original, MIT.

A workflow for planning controlled comparisons between two or more versions of something, such as a webpage or message. A/B testing measures which version performs better using pre-set rules.

In plain words
What is it for?
Use it to define sample sizes, test duration, success measures, stopping rules, and repeatable experiment plans.
Why use it?
It reduces the risk of calling a version better because of random short-term results or incomplete data.

Skill for Claude CodeCodex

Written for Claude Code and Codex: shipped in a Claude Code plugin, but also agents/openai.yaml present.

Part of the suede-marketing plugin — 40 skills shipped together , and of suede-skills

Good fit Use it to define sample sizes, test duration, success measures, stopping rules, and repeatable experiment plans.

Compare 6 skills from other repositories ↓
Install with agentmods
npx agentmods add skills/jasoncolapietro/suede-creator-skills/suede-ab-testing
Install

Getting it into your agent

One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.

Any agent
npx skills add JasonColapietro/suede-creator-skills --skill suede-ab-testing
Clone the repo
git clone --depth 1 https://github.com/JasonColapietro/suede-creator-skills

Made for: Claude Code, Codex.

Or install suede-marketing, the plugin that ships this one along with the rest of its 40 skills.

Wrote this? Show the measurements

A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.

agentmods badge for suede-ab-testing

README.md
[![agentmods](https://agentmods.dev/badge/skills/jasoncolapietro/suede-creator-skills/suede-ab-testing/github.svg)](https://agentmods.dev/skills/jasoncolapietro/suede-creator-skills/suede-ab-testing)
Your own site
<a href="https://agentmods.dev/skills/jasoncolapietro/suede-creator-skills/suede-ab-testing"><img src="https://agentmods.dev/badge/skills/jasoncolapietro/suede-creator-skills/suede-ab-testing/github.svg" alt="Measured on agentmods" height="20"></a>

Or the 80×15 button, for a site that already has a row of RSS and ATOM ones. Only the verdict fits; the numbers stay here.

agentmods 80×15 button for suede-ab-testing

Your own site · 80×15
<a href="https://agentmods.dev/skills/jasoncolapietro/suede-creator-skills/suede-ab-testing"><img src="https://agentmods.dev/badge/skills/jasoncolapietro/suede-creator-skills/suede-ab-testing.svg" alt="Reviewed on agentmods" width="80" height="20"></a>
Per session 88 Skills are progressive disclosure: only the name and description are preloaded; the body loads when the skill is used.
When invoked 3,367 The whole file, excluding the scripts and references it only reads on demand.
Security scan A 0 findings. A grade says what 26 rules found in the file — not that it is safe. Third-party audits
  • NVIDIA SkillSpector pass 7 Sept 2026
How audits are shown
Origin original No closer match found in the catalogue.
Token cost

What it costs to keep this loaded

Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.

ModelPer sessionOnce invoked
Fable 5.1 $0.00088 $0.03367
Opus 5.5 $0.00035 $0.01347
Sonnet 5.5 $0.00018 $0.00673
Haiku 4.5 $0.00009 $0.00337

Measured today against content hash 7a76852328c8, method: parsed. Prices are Anthropic first-party input rates as of 2026-10-07, from the pricing page.

Security

Grade A, and why

suede-ab-testing scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured today.

A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.

Nothing flagged

None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.

skills/suede-ab-testing/SKILL.md · 369 lines

How it starts

The opening of the file, as written. The whole thing — 369 lines — stays where its author put it; the contents beside it link to each section on GitHub.

Suede AI A/B Testing Skills

Read the worked delivery contract for a concrete input, expected artifact, failure response and validation example.

Compatibility

Works from supplied experiment data and product context; file access for saved plans and a calculator/code runtime for power or statistical calculations. Analytics access is optional when exports are provided.

Use this experimentation playbook to design tests that produce statistically valid, actionable results.

The Iron Law

Predeclare three things before a test launches: sample per variant,
minimum duration, and the decision rule: and read the result only once
all three are satisfied. A result read before then is preliminary.
Never a winner.
  • Sample per variant: the Sample Size table below, or a calculator run on your actual baseline.
  • Minimum duration: 1 full week (day-of-week variation), 2 business cycles (B2B), through paydays (e-commerce): see the "Minimum Duration Rules" section of references/sample-size-guide.md.
  • Decision rule: which metric, at which threshold, decides the call: written down before launch, not after.

Two carve-outs, and only these two:

  • A predeclared sequential or always-valid design may look early under its own stopping rule (see "Sequential Testing" in the sample-size guide). Declaring it sequential after the peek does not count.
  • A guardrail-triggered stop for harm is a stop, not a winner call. Kill the variant, report no result.

Initial Assessment

Check for .agents/product-marketing.md (or .claude/product-marketing.md, or the legacy product-marketing-context.md) and read it if present, baseline conversion rate, traffic volume, and available tooling decide whether a test is even powerable, and they are usually already written down there.

Then work the intake list under Task-Specific Questions below; ask only what the context file did not already answer.


Read the full file on GitHub · 369 lines

Files

What ships with it

6 files beside SKILL.md in the same directory: the scripts, references and assets a skill reads on demand. Not counted in the per-session cost; read them before you install if any of them is executable.

Changes

What this file has done since we first saw it

Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.

  1. today Changed · +8 lines · -23 tokens per session 7a76852328c8
  2. 3d ago Changed · +2 lines · +31 tokens per session bf7924cb009f
  3. 23d ago Changed f2effe4f27bb
  4. 1mo ago First seen · 359 lines · 80 tokens per session scan A 66cd6faf6653

Subscribe to this mod's changes

suede-ab-testing is a skill published in the GitHub repository JasonColapietro/suede-creator-skills (126 stars, last pushed today), licensed MIT. It adds 88 tokens to every session and 3,367 once invoked, about $0.0004 per session on Opus 5.5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-30.

Related

Other skills, from other repositories

skill-router

Use when a [skill-router] route card appears in the turn, when no card appeared on a non-trivial task, or when a route looks wrong. Routes every prompt to the right installed skill, pairs it with a process skill, tiers enforcement, and briefs sub-agents. 79 local skills + plugin skills indexed.

hussi9/skill-router · 69 tokens

fable-commander

A planning and review workflow in which one model prepares decisions while selected models handle research and execution. It is written for Fable and Claude Code, tools for coordinating multi-step software work.

DennisWei9898/fable-commander · 176 tokens

tao

Coordinate multi-stage engineering work with cost-aware Lead execution, reusable independent Workers, repository-backed handoffs, and bounded workspace housekeeping. Activate only for an explicit $tao command; maintenance of this skill never activates orchestration.

jaywavfeng/tiered-agent-orchestrator · 47 tokens

manara

Orchestrate a software task end-to-end in Claude Code. Use this whenever the user describes building, fixing, reviewing, planning, or shipping something in a project — e.g. "build the X MVP", "add feature Y", "review this PR", "diagnose this bug", "plan this out" — and especially when resuming work after a context…

OmerWafaey/manara · 148 tokens

code-conventions

General code conventions that keep a codebase predictable — reversible side effects, switching on discriminant tags, explicit-over-implicit at boundaries, configuration over hardcoded tunables, branded opaque ids, trusting types at typed boundaries, and honest empty catches. Use when writing, reviewing, or refactoring…

arch3rPro/dsh-skills · 81 tokens

defensive-patterns

Defensive-programming patterns to apply when writing or reviewing lifecycle, concurrency, subprocess, teardown, error-reporting, or untrusted-IO code. Use when a change touches async setup or teardown, callbacks, spawned processes, workers, event listeners, disposal, error aggregation, temp files, or anything…

arch3rPro/dsh-skills · 81 tokens