Getting it into your agent
One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.
npx skills add sergeyizmailov/knowledge-delta-skills --skill measurement-experimentation-opsgit clone --depth 1 https://github.com/sergeyizmailov/knowledge-delta-skillsWrote this? Show the measurements
A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.
[](https://agentmods.dev/skills/sergeyizmailov/knowledge-delta-skills/measurement-experimentation-ops)<a href="https://agentmods.dev/skills/sergeyizmailov/knowledge-delta-skills/measurement-experimentation-ops"><img src="https://agentmods.dev/badge/skills/sergeyizmailov/knowledge-delta-skills/measurement-experimentation-ops/github.svg" alt="Measured on agentmods" height="20"></a>Or the 80×15 button, for a site that already has a row of RSS and ATOM ones. Only the verdict fits; the numbers stay here.
<a href="https://agentmods.dev/skills/sergeyizmailov/knowledge-delta-skills/measurement-experimentation-ops"><img src="https://agentmods.dev/badge/skills/sergeyizmailov/knowledge-delta-skills/measurement-experimentation-ops.svg" alt="Reviewed on agentmods" width="80" height="20"></a>What it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5.1 | $0.00101 | $0.01036 |
| Opus 5 | $0.00051 | $0.00518 |
| Sonnet 5 | $0.00020 | $0.00207 |
| Haiku 4.5 | $0.00010 | $0.00104 |
Grade A, and why
measurement-experimentation-ops scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 7d ago.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Nothing flagged
None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.
How it starts
The opening of the file, as written. The whole thing — 69 lines — stays where its author put it; the contents beside it link to each section on GitHub.
Measurement & Experimentation Ops
The other skills act on measured differences; this decides whether a difference is real or noise before they do.
Pick the testing mode by the decision at stake
Three modes, different evidence bars — match to the cost of being wrong:
- Causal: estimates incrementality (no design proves causality without assumptions). Two sub-modes — a randomized experiment (one treatment vs a non-overlapping control/holdout, pre-sized) is the strongest; quasi-experimental causal estimation (GeoLift synthetic control, pre/post) is the fallback when you can't randomize. Use for expensive, hard-to-reverse bets: offer, funnel, landing page, "does this channel even lift sales." Cost: volume + discipline + often a Meta rep.
- Screening (directional): many concepts in one ad set / parallel ABO cells; delivery is UNEQUAL by design, so a "winner" is a hypothesis, not a proof. Use for high-throughput creative hunting where being fast beats being certain. Never present a screen result as validated.
- Infrastructure (isolate infra variance): hold the CREATIVE fixed, vary one infra axis (domain / proxy cluster / account batch) across a balanced set to attribute delivery/ban/CPM differences to infra, not creative. The grey inversion of a normal test — see meta-grey-ops/06.
Feasibility gate (grey reality — check BEFORE promising a clean test)
Causal measurement often isn't available on grey/small-account buys: too little volume to power a holdout, accounts die mid-test, no clean pixel signal, no rep for a sandboxed Conversion Lift. When you can't run causal, SAY SO and drop to the best affordable proxy (geo holdout, pre/post with tracker truth, screening) — label it directional, don't dress a screen up as a lift study. Choosing the honest weaker method beats a "causal" test that's silently contaminated.
Validity traps (each one silently flips a conclusion)
- SRM: check the RANDOMIZED-UNIT split (a 50/50 arriving 55/45 = broken randomization/logging → invalid) — on assignment counts, NOT on spend/impressions/conversions (those diverging is a delivery effect, not SRM).
- Peeking: Meta's A/B "end test early if a winner is found" — leave off and run the pre-set window unless Meta's sequential rule is verified (unpublished) (02).
- Contamination: overlapping audiences between cells — Advantage+ broad bleeding into manual cells; duplicated winners cannibalizing in the auction → not clean groups. Use the A/B tool's non-overlapping split, or geo separation.
- Conversion lag: judging before the payout event matures counts spend against unripe conversions → every fresh cohort looks like a loser. Window ≥ lag; nowcast if you must decide early (tracker-ops/03).
- Multiple testing: screening tolerates chance winners (you re-test anyway); a causal decision needs the bar corrected for the number of comparisons.
- Underpowered: "no significant difference" ≠ "no effect" — size first (01).
What ships with it
3 files beside SKILL.md in the same directory: the scripts, references and assets a skill reads on demand. Not counted in the per-session cost; read them before you install if any of them is executable.
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- 7d ago Changed · -12 lines cd62c4b59ac9
- 11d ago First seen · 81 lines · 101 tokens per session scan A 347473de01f1
measurement-experimentation-ops is a skill published in the GitHub repository sergeyizmailov/knowledge-delta-skills (3 stars, last pushed 4d ago), licensed MIT. It adds 101 tokens to every session and 1,036 once invoked, about $0.0005 per session on Opus 5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-31.
Other skills, from other repositories
testing
Verify a Skyvern deployment is working correctly by smoke-testing the backend API, frontend rendering, browser session provisioning, and workflow execution. Use when the user says 'is Skyvern working', 'test my deployment', 'verify the installation', 'smoke test', or needs to check that a self-hosted or local Skyvern…
screen-reader-testing
Test web applications with screen readers including VoiceOver, NVDA, and JAWS. Use when validating screen reader compatibility, debugging accessibility issues, or ensuring assistive technology support.
mem0-test-integration
Verify a Mem0 integration produced by /mem0-integrate. Runs in the same workspace on the same branch (loose coupling) — installs dependencies, runs the repo's native test suite, then exercises a real end-to-end smoke flow against the user's API key. Produces a scorecard. TRIGGER when: user has just run /mem0-integrate…
langbot-dev
Develop, build, and debug the LangBot core backend and web frontend. Use when working inside the LangBot repository — backend (Python/Quart, src/langbot/pkg), the Vite/React web UI, HTTP API controllers/services, Alembic migrations, or the MCP server. Covers the dev environment (uv, pnpm), repo layout, the API auth…
qa
QA test your code changes by reading your git diff, choosing the right validation path for frontend/browser and backend changes, and reporting pass/fail with evidence.
patrol-e2e-testing
Use when writing E2E/integration tests, testing native interactions like permissions or system dialogs, capturing UI regressions, or validating cross-platform behavior (Patrol 4.x).