experimentation-analytics

experimentation-analytics is a skill for Claude Code, Codex from rampstackco/claude-skills. It costs 179 tokens per session (7,295 once invoked), scanned A, original, MIT.

A guide to interpreting the results of controlled experiments, where different groups receive different versions and their outcomes are compared.

In plain words
What is it for?
Use it when deciding whether to launch, stop, or revise an experiment, including results involving confidence intervals, p-values, ratios, repeated testing, or different user groups.
Why use it?
It helps avoid incorrect decisions caused by misleading uncertainty, repeated checking, multiple comparisons, or poorly interpreted metrics.

Skill for Claude CodeCodex

Part of the rampstack-skills plugin — 103 skills shipped together

Install

Getting it into your agent

One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.

agentmods
npx agentmods add skills/rampstackco/claude-skills/experimentation-analytics
Any agent
npx skills add rampstackco/claude-skills --skill experimentation-analytics
Clone the repo
git clone --depth 1 https://github.com/rampstackco/claude-skills

Made for: Claude Code, Codex.

Or install rampstack-skills, the plugin that ships this one along with the rest of its 103 skills.

Wrote this? Show the measurements

A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.

agentmods badge for experimentation-analytics

README.md
[![agentmods](https://agentmods.dev/badge/skills/rampstackco/claude-skills/experimentation-analytics.svg)](https://agentmods.dev/skills/rampstackco/claude-skills/experimentation-analytics)
Your own site
<a href="https://agentmods.dev/skills/rampstackco/claude-skills/experimentation-analytics"><img src="https://agentmods.dev/badge/skills/rampstackco/claude-skills/experimentation-analytics.svg" alt="Measured on agentmods" height="20"></a>
Per session 179 Skills are progressive disclosure: only the name and description are preloaded; the body loads when the skill is used.
When invoked 7,295 The whole file, excluding the scripts and references it only reads on demand.
Security scan A 0 findings. Scan, not verified.
Origin original No closer match found in the catalogue.
Token cost

What it costs to keep this loaded

Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.

ModelPer sessionOnce invoked
Fable 5.1 $0.00179 $0.07295
Opus 5 $0.00089 $0.03648
Sonnet 5 $0.00036 $0.01459
Haiku 4.5 $0.00018 $0.00730

Measured 6d ago against content hash 6b84055d9897, method: parsed. Prices are Anthropic first-party input rates as of 2026-09-05, from the pricing page.

Security

Grade A, and why

experimentation-analytics scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 6d ago.

A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.

Nothing flagged

None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.

skills/experimentation-analytics/SKILL.md · 334 lines

How it starts

The opening of the file, as written. The whole thing — 334 lines — stays where its author put it; the contents beside it link to each section on GitHub.

Experimentation Analytics

A data-team-mentor's playbook for interpreting experiment results without fooling yourself.

The result panel is the moment-of-truth for an experiment. The numbers on it determine whether you ship, kill, or iterate. They also expose every shortcut taken in the design phase: an underpowered test produces wide confidence intervals; a peeked test produces a too-narrow p-value; a ratio metric without delta-method correction produces overconfident lift estimates. Most ship-the-wrong-thing decisions trace back to misreading the result panel.

This skill is the discipline that prevents misreading. It assumes the experiment was designed well (see the experiment-design skill). It assumes the platform's results panel is technically correct (most modern platforms are; some older ones are not). It assumes you can read a number off a screen. The hard part is knowing what each number actually means and what it does not, and that is what is here.

When to use this skill: any time you are reading an experiment result panel and about to make a ship, kill, or iterate decision.


What this skill is for

This skill covers result interpretation, the statistical concepts that make the numbers trustworthy, and the dashboard reconciliation work that prevents executive-level confusion when the experiment number does not match the BI number. The audience is product managers and data analysts who read experiment results together and need a shared vocabulary that does not paper over the dangerous parts of statistics.

Companion skills cover the adjacent territory. The experiment-design skill covers pre-experiment thinking: hypothesis, sample size, MDE, segments, what NOT to test. Read it before designing the test; read this skill when reading the result. The feature-flagging skill covers the operational mechanics of flag management, environment promotion, and stale-flag cleanup. Together the three skills span the experimentation lifecycle from intent through interpretation. For platform-specific MCP commands, consult the chosen platform's docs; Statsig, PostHog, Optimizely, GrowthBook, Eppo, Amplitude, and Kameleoon all expose rich analytics surfaces that this skill informs how to read.

Read the full file on GitHub · 334 lines

Changes

What this file has done since we first saw it

Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.

  1. 6d ago First seen · 334 lines · 179 tokens per session scan A 6b84055d9897

Subscribe to this mod's changes

experimentation-analytics is a skill published in the GitHub repository rampstackco/claude-skills (817 stars, last pushed 8d ago), licensed MIT. It adds 179 tokens to every session and 7,295 once invoked, about $0.0009 per session on Opus 5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-30.