data-warehouse-experimentation

data-warehouse-experimentation is a skill for Claude Code from rampstackco/claude-skills. It costs 157 tokens per session (6,083 once invoked), scanned A, original, MIT.

A guide to running controlled experiments using an existing data warehouse, such as BigQuery or Snowflake, instead of a dedicated experiment platform. It covers assigning users, recording exposure, defining metrics, and analysing results with SQL or Python.

In plain words
What is it for?
Use it to design warehouse-based A/B tests, define metrics in dbt models, analyse results, and apply methods such as CUPED or sequential testing.
Why use it?
It explains how to use the data stack you already operate while handling the tracking and statistical work an experiment platform normally provides.

Skill for Claude Code

Written for Claude Code: shipped in a Claude Code plugin.

Part of the rampstack-skills plugin — 103 skills shipped together

Good fit Use it to design warehouse-based A/B tests, define metrics in dbt models, analyse results, and apply methods such as CUPED or sequential testing.

Compare 6 skills from other repositories ↓
Install with agentmods
npx agentmods add skills/rampstackco/claude-skills/data-warehouse-experimentation
Install

Getting it into your agent

One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.

Any agent
npx skills add rampstackco/claude-skills --skill data-warehouse-experimentation
Clone the repo
git clone --depth 1 https://github.com/rampstackco/claude-skills

Made for: Claude Code.

Or install rampstack-skills, the plugin that ships this one along with the rest of its 103 skills.

Wrote this? Show the measurements

A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.

agentmods badge for data-warehouse-experimentation

README.md
[![agentmods](https://agentmods.dev/badge/skills/rampstackco/claude-skills/data-warehouse-experimentation.svg)](https://agentmods.dev/skills/rampstackco/claude-skills/data-warehouse-experimentation)
Your own site
<a href="https://agentmods.dev/skills/rampstackco/claude-skills/data-warehouse-experimentation"><img src="https://agentmods.dev/badge/skills/rampstackco/claude-skills/data-warehouse-experimentation.svg" alt="Measured on agentmods" height="20"></a>
Per session 157 Skills are progressive disclosure: only the name and description are preloaded; the body loads when the skill is used.
When invoked 6,083 The whole file, excluding the scripts and references it only reads on demand.
Security scan A 0 findings. A grade says what 26 rules found in the file — not that it is safe. Third-party audits
  • NVIDIA SkillSpector pass 7 Sept 2026
How audits are shown
Origin original No closer match found in the catalogue.
Token cost

What it costs to keep this loaded

Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.

ModelPer sessionOnce invoked
Fable 5.1 $0.00157 $0.06083
Opus 5 $0.00078 $0.03041
Sonnet 5 $0.00031 $0.01217
Haiku 4.5 $0.00016 $0.00608

Measured 8d ago against content hash e3aa63517c34, method: parsed. Prices are Anthropic first-party input rates as of 2026-09-07, from the pricing page.

Security

Grade A, and why

data-warehouse-experimentation scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 8d ago.

A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.

Nothing flagged

None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.

skills/data-warehouse-experimentation/SKILL.md · 396 lines

How it starts

The opening of the file, as written. The whole thing — 396 lines — stays where its author put it; the contents beside it link to each section on GitHub.

Data Warehouse Experimentation

A senior data scientist's playbook for running experiments natively out of BigQuery, Snowflake, or any modern data warehouse, with metric definitions in dbt and statistical analysis in SQL or Python.

Most companies that run experiments at scale use a dedicated platform. Statsig, Optimizely, LaunchDarkly with experimentation, PostHog, Amplitude Experiment. The platforms are good. They handle assignment, instrumentation, and analysis in one product, and the SQL-savvy data team does not have to reinvent the variance reduction wheel.

There is a different operational model that mature data teams increasingly choose: warehouse-native experimentation. Assignment happens in code or via feature flags. Exposure events fire to the warehouse like any other event. Metrics are defined as dbt models. Statistical analysis runs as SQL or in a Python notebook against warehouse data. The "experiment platform" is just your existing data stack.

This skill covers when warehouse-native is the right call, the architecture, and the specific techniques that make it work: assignment patterns, exposure logging discipline, metric definitions in dbt, t-tests and CUPED in SQL, sequential testing, and the pitfalls that take down homegrown setups.

When to use this skill: deciding between platform vs warehouse-native, building a warehouse-native experiment infrastructure, auditing an existing one, or running a specific experiment when the platform of record cannot handle a custom metric or segmentation.


What this skill is for

This skill spans the operational execution model for warehouse-native experimentation. It does not replace the methodology and interpretation skills; it composes with them.

  • experiment-design covers methodology: hypotheses, sample size, randomization unit, primary metric. Tool-agnostic. Read it first to design the experiment correctly regardless of where it runs.
  • experimentation-analytics covers interpretation: confidence intervals, p-values, effect size, decision frameworks. Tool-agnostic. Read it when results land.
  • experimentation-platform-orchestrator covers the platform-vs-warehouse decision in detail. Read it to decide whether to use a platform or this skill.
  • feature-flagging covers assignment infrastructure when not running through a platform. Read it for the flag-management discipline that this skill assumes.
  • This skill (data-warehouse-experimentation) covers the operational execution: SQL-based assignment, exposure logging, metric definitions in dbt, statistical analysis in SQL or Python, variance reduction, sequential testing.

Read the full file on GitHub · 396 lines

Changes

What this file has done since we first saw it

Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.

  1. 8d ago First seen · 396 lines · 157 tokens per session scan A e3aa63517c34

Subscribe to this mod's changes

data-warehouse-experimentation is a skill published in the GitHub repository rampstackco/claude-skills (826 stars, last pushed 10d ago), licensed MIT. It adds 157 tokens to every session and 6,083 once invoked, about $0.0008 per session on Opus 5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-30.

Related

Other skills, from other repositories

drug-discovery

Drug discovery: ChEMBL search, drug-likeness, interactions.

NousResearch/hermes-agent · 19 tokens

aaai-experiments

Use when designing or auditing AAAI experiments for the broad-AI program committee, including baselines, ablations, statistical significance, robustness, human evaluation, AI-for-Social-Impact and alignment/safety evidence, compute and cost reporting, and reproducibility-checklist alignment for Phase-1 survival.

brycewang-stanford/Awesome-Journal-Skills · 66 tokens

acl-experiments

Use when designing or auditing experiments for an ACL paper, covering tuned LLM baselines, multi-dataset and multilingual evaluation, statistical significance and variance, human evaluation with agreement reporting, contamination and prompt-sensitivity controls, ablations, and error-analysis expectations in NLP…

brycewang-stanford/Awesome-Journal-Skills · 59 tokens

acmmm-experiments

Use when designing or auditing the experiments of an ACM MM (ACM Multimedia) paper — matched baselines per modality, ablations that isolate the cross-modal fusion, user studies or QoE measurement where the claim is subjective, dataset and media licensing, and honest compute reporting, so evidence supports a multimedia…

brycewang-stanford/Awesome-Journal-Skills · 68 tokens

case-interview-practice

Interactive consulting case interview practice with structured frameworks, feedback mechanisms, and progressive difficulty. Use when preparing for management consulting interviews, case competitions, or business problem-solving exercises.

travisjneuman/.claude · 39 tokens

i18n-localization

Internationalization and localization for global applications. Use when adding multi-language support, handling regional formats, or preparing apps for global markets.

travisjneuman/.claude · 32 tokens