experiment

If the skill install path cannot be resolved (some sandboxed environments): read the script file(s) from this skill, write a copy into a scripts/ folder inside the working folder, and run from there. The scripts are self-contained.

Skill: /experiment — OpenXP Experimentation Platform

Purpose

Multi-mode skill for the full experiment lifecycle — from design through analysis to ship/no-ship decision. Orchestrates the experiment plugin agents and calls the coded statistical library bundled in this skill at scripts/experiment_stats/ instead of improvising Python.

Using the bundled library: add this skill's scripts/ directory to sys.path, then import, e.g.

import sys
sys.path.insert(0, "<path to this skill>/scripts")  # the scripts/ dir next to this SKILL.md
from experiment_stats import srm_check

Requires pandas, numpy, and scipy; power.py and corrections.py also need statsmodels (install it in the sandbox if missing).

When to Use

Invoke as /experiment [mode] or trigger on experiment-related intents:

• "I want to run an experiment"
• "Analyze this A/B test"
• "Did this experiment work?"
• "What's the power for this test?"

Modes

/experiment design

Purpose: Create a pre-registered experiment config. Agent: the experiment-designer plugin agent Flow:

1. Run Experiment Brief skill to capture hypothesis, north star, guardrails
2. Invoke Experiment Designer agent
3. Output: experiments/{slug}/experiment.yaml (from templates/experiment.yaml) Checkpoint: Config review (Type B — skippable with --just-do-it)

/experiment power

Purpose: Power analysis + duration estimation. Flow:

1. Read experiments/{slug}/experiment.yaml for metric type, baseline, MDE
2. Call the bundled power module (scripts/experiment_stats/power.py):
   • Proportion metric → power_proportion(baseline_rate, mde)
   • Continuous metric → power_mean(baseline_mean, baseline_std, mde)
3. Call duration_estimate(total_sample, daily_traffic, allocation)
4. Update experiment.yaml with computed values (sample_size, duration, viable)
5. If NOT_VIABLE → suggest /causal select as alternative Checkpoint: Power viability (Type C — NOT_VIABLE fires mandatory checkpoint)