creating-experiments

creating-experiments is a skill for Claude Code, Codex from PostHog/posthog-foss. It costs 96 tokens per session (1,260 once invoked), scanned A, original, MIT.

A guide for creating a new PostHog experiment, or A/B test, from its hypothesis through rollout and measurement setup.

In plain words
What is it for?
It helps define an experiment name, hypothesis, feature-flag key, rollout settings, and analytics configuration.
Why use it?
It gives the experiment a clear structure so the team can start with a draft and refine its audience, variants, and success measures.

Skill for Claude CodeCodex

Written for no agent in particular: nothing here depends on one.

Good fit It helps define an experiment name, hypothesis, feature-flag key, rollout settings, and analytics configuration.

Compare 6 skills from other repositories ↓
Install with agentmods
npx agentmods add skills/posthog/posthog-foss/creating-experiments
Install

Getting it into your agent

One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.

Any agent
npx skills add PostHog/posthog-foss --skill creating-experiments
Clone the repo
git clone --depth 1 https://github.com/PostHog/posthog-foss

Made for: Claude Code, Codex.

Wrote this? Show the measurements

A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.

agentmods badge for creating-experiments

README.md
[![agentmods](https://agentmods.dev/badge/skills/posthog/posthog-foss/creating-experiments/github.svg)](https://agentmods.dev/skills/posthog/posthog-foss/creating-experiments)
Your own site
<a href="https://agentmods.dev/skills/posthog/posthog-foss/creating-experiments"><img src="https://agentmods.dev/badge/skills/posthog/posthog-foss/creating-experiments/github.svg" alt="Measured on agentmods" height="20"></a>

Or the 80×15 button, for a site that already has a row of RSS and ATOM ones. Only the verdict fits; the numbers stay here.

agentmods 80×15 button for creating-experiments

Your own site · 80×15
<a href="https://agentmods.dev/skills/posthog/posthog-foss/creating-experiments"><img src="https://agentmods.dev/badge/skills/posthog/posthog-foss/creating-experiments.svg" alt="Reviewed on agentmods" width="80" height="20"></a>
Per session 96 Skills are progressive disclosure: only the name and description are preloaded; the body loads when the skill is used.
When invoked 1,260 The whole file, excluding the scripts and references it only reads on demand.
Security scan A 0 findings. A grade says what 26 rules found in the file — not that it is safe. Third-party audits
  • NVIDIA SkillSpector pass 7 Sept 2026
How audits are shown
Origin original No closer match found in the catalogue.
Token cost

What it costs to keep this loaded

Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.

ModelPer sessionOnce invoked
Fable 5.1 $0.00096 $0.01260
Opus 5 $0.00048 $0.00630
Sonnet 5 $0.00019 $0.00252
Haiku 4.5 $0.00010 $0.00126

Measured 9d ago against content hash e513651cdc65, method: parsed. Prices are Anthropic first-party input rates as of 2026-09-12, from the pricing page.

Security

Grade A, and why

creating-experiments scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 9d ago.

A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.

Nothing flagged

None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.

products/experiments/skills/creating-experiments/SKILL.md · 109 lines

How it starts

The opening of the file, as written. The whole thing — 109 lines — stays where its author put it; the contents beside it link to each section on GitHub.

Creating experiments

This skill walks through the 3-step flow for creating a new A/B test experiment.

Core principle: draft first, iterate on details

Create the experiment as a draft quickly, then iterate on metrics and configuration. The user gets a tangible draft immediately and can refine it.

The 3-step creation flow

Step 1: What are we testing?

Gather these before calling experiment-create:

  • Experiment name — descriptive, inferred from context when possible
  • Hypothesis — what you expect to happen (goes in description)
  • Feature flag key — kebab-case. Ask if they want a new flag or to reuse an existing one. The flag is auto-created — do NOT create one separately.
  • Type — leave empty (will internally default to "product". The "web" value is reserved for no-code experiments configured visually with the PostHog toolbar in a browser; it cannot be meaningfully driven via MCP. If a user asks for a no-code/toolbar experiment, point them to the PostHog UI instead of creating one here.)

If the user gives enough context to infer these, don't ask — just proceed.

Step 2: Who sees what variant?

This is about rollout configuration.

Before asking any rollout question, load configuring-experiment-rollout. The disambiguation wording, recommendations, and post-answer branches live there — do not formulate rollout questions yourself, and do not assume an example you remember covers the user's path.

Key decision points (covered in detail by configuring-experiment-rollout):

  • Variant split (how many variants, what percentage each)
  • Overall rollout percentage (what % of all users enter the experiment)
  • Whether to persist the flag across authentication steps

If the user doesn't mention rollout specifics, use defaults: 50/50 control/test, 100% rollout.

Step 3: How to measure impact?

This is about analytics and metrics. Load the configuring-experiment-analytics skill for guidance. That skill's first step checks for an existing shared metric to reuse before building a new one — don't duplicate a metric the project already has set up.

Read the full file on GitHub · 109 lines

Changes

What this file has done since we first saw it

Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.

  1. 9d ago First seen · 109 lines · 96 tokens per session scan A e513651cdc65

Subscribe to this mod's changes

creating-experiments is a skill published in the GitHub repository PostHog/posthog-foss (715 stars, last pushed today), licensed MIT. It adds 96 tokens to every session and 1,260 once invoked, about $0.0005 per session on Opus 5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-09-03.

Related

Other skills, from other repositories

systematic-debugging

Use when encountering any bug, test failure, or unexpected behavior, before proposing fixes.

obra/superpowers · 21 tokens

local-ai-agents

Build local-first AI agents that run entirely on a developer workstation with Microsoft Foundry Local and Qwen function-calling models. Covers Small Language Models (SLMs), the OpenAI-compatible local endpoint, sandboxed local tools, local RAG with Chroma, local MCP servers, hybrid cloud/local routing, and the…

microsoft/ai-agents-for-beginners · 200 tokens

next-cache-components-adoption

Turn on Cache Components in a Next.js app and resolve the blocking routes it surfaces. Use when the user wants to enable, adopt, or migrate to Cache Components, flip the cacheComponents flag, work through a flood of blocking-prerender / instant validation errors, run the cache-components-instant-false codemod, or…

vercel/next.js · 95 tokens

chat-pet-sprite-creation

Use when creating or changing VS Code chat pet sprite art, sprite sheets, state animations, eye treatments, Stable/Insiders variants, or pet transitions under src/vs/workbench/contrib/chat/browser/widget/media/chatPet.

microsoft/vscode · 53 tokens

cpu-profile-analysis

Analyze V8/Chrome CPU profiles (.cpuprofile) and DevTools trace files (Trace-.json). Use when: profiling performance, investigating slow functions, comparing code paths, finding bottlenecks, analyzing timeToRequest, understanding call trees from sampling profiler data, analyzing layout/paint/rendering, investigating…

microsoft/vscode · 71 tokens

insight-error-page

Write or audit an insight-kind error page for the Next.js dev overlay. Use when creating a new errors/ .mdx page, auditing an existing one, or checking that a page matches the framework fix cards. Covers page structure, title alignment, FixCard cards with Copy prompt button, code snippets, terminology verification…

vercel/next.js · 83 tokens