What is experiment-reproducibility?

A workflow for making computer-based experiments repeatable by recording the environment, code version, data, settings, hardware, random seeds, and results for each run.

Use it to pin software environments, define experiment settings, record run details, aggregate results, and regenerate tables and figures.

It lets an examiner or the researcher reproduce a result later instead of relying on an undocumented one-off run.

experiment-reproducibility — Skill for Claude Code by KhaledSaeed18

Install

Getting it into your agent

One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.

Any agent

npx skills add KhaledSaeed18/dotclaude --skill experiment-reproducibility

Clone the repo

git clone --depth 1 https://github.com/KhaledSaeed18/dotclaude

Made for: Claude Code.

Or install thesis, the plugin that ships this one along with the rest of its 21 skills, 3 commands, 2 agents.

Wrote this? Show the measurements

A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.

README.md

[![agentmods](https://agentmods.dev/badge/skills/khaledsaeed18/dotclaude/experiment-reproducibility/github.svg)](https://agentmods.dev/skills/khaledsaeed18/dotclaude/experiment-reproducibility)

Your own site

<a href="https://agentmods.dev/skills/khaledsaeed18/dotclaude/experiment-reproducibility"><img src="https://agentmods.dev/badge/skills/khaledsaeed18/dotclaude/experiment-reproducibility/github.svg" alt="Measured on agentmods" height="20"></a>

Or the 80×15 button, for a site that already has a row of RSS and ATOM ones. Only the verdict fits; the numbers stay here.

Your own site · 80×15

<a href="https://agentmods.dev/skills/khaledsaeed18/dotclaude/experiment-reproducibility"><img src="https://agentmods.dev/badge/skills/khaledsaeed18/dotclaude/experiment-reproducibility.svg" alt="Reviewed on agentmods" width="80" height="20"></a>

Per session 96 Skills are progressive disclosure: only the name and description are preloaded; the body loads when the skill is used.

When invoked 1,269 The whole file, excluding the scripts and references it only reads on demand.

Security scan A 0 findings. A grade says what 26 rules found in the file — not that it is safe.

Origin original No closer match found in the catalogue.

Token cost

What it costs to keep this loaded

Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.

Model	Per session	Once invoked
Fable 5.1	$0.00096	$0.01269
Opus 5.5	$0.00038	$0.00508
Sonnet 5.5	$0.00019	$0.00254
Haiku 4.5	$0.00010	$0.00127

Measured 25d ago against content hash 19f966538c22, method: parsed. Prices are Anthropic first-party input rates as of 2026-10-07, from the pricing page.

Security

Grade A, and why

experiment-reproducibility scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 25d ago.

A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.

Nothing flagged

None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.

.claude-plugin/plugins/thesis/skills/experiment-reproducibility/SKILL.md · 77 lines

How it starts

The opening of the file, as written. The whole thing — 77 lines — stays where its author put it; the contents beside it link to each section on GitHub.

A result that cannot be regenerated from a command is an anecdote. Reproducibility is cheap at the start and impossible at the end, so this is set up before the first run that will appear in the thesis.

Layout

experiments/
  env/          environment.yml or requirements.txt with exact pins; Dockerfile if system deps matter
  configs/      one YAML per experiment: exp01-baseline.yaml, exp02-ours.yaml
  data/         raw/ (never modified), processed/ (generated by scripts), DATA.md (source, licence, version, checksum)
  src/          the code; run.py takes --config and --seed
  runs/         one directory per run: <exp>-<seed>-<timestamp>/ with manifest.json, metrics.json, log.txt
  results/      aggregated tables (CSV) generated from runs/ by aggregate.py
  figures/      generated from results/ by figure scripts (research-figures)
  Makefile      or justfile: `make exp01` runs all seeds; `make results` aggregates; `make figures`

Everything under runs/, results/, and figures/ is generated; regenerate rather than edit. Commit results/ and figures/ (they are small) so the thesis builds from a clean clone; do not commit runs/ unless small (use DVC or a release asset otherwise).

Environment

Python: conda env export --no-builds or pip freeze into env/, with the Python version. Better: pyproject.toml with a lockfile (uv lock, poetry lock).
Record the CUDA/driver version and the exact framework build when GPUs are involved; results differ across them.
A Dockerfile when anything outside the language environment matters (system libraries, a database).
The manifest records python --version, pip freeze hash, nvidia-smi output, and uname -a at run time regardless.

Determinism

One --seed argument that seeds every source: random, numpy, torch (and torch.cuda.manual_seed_all), tf, data shuffling, and worker init. Use a helper seed_everything(seed) called first.
Set deterministic flags where the framework offers them (torch.use_deterministic_algorithms(True), CUBLAS_WORKSPACE_CONFIG) and record whether they were on; some ops stay nondeterministic on GPU and the thesis should say so.
Run every experiment with at least three seeds (five or ten when cheap); report mean and SD or CI (statistics-advisor). A single-seed number is not a result.

Read the full file on GitHub · 77 lines

Changes

What this file has done since we first saw it

Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.

25d ago First seen · 77 lines · 96 tokens per session scan A 19f966538c22

Subscribe to this mod's changes

experiment-reproducibility is a skill published in the GitHub repository KhaledSaeed18/dotclaude (5 stars, last pushed 2d ago), licensed MIT. It adds 96 tokens to every session and 1,269 once invoked, about $0.0004 per session on Opus 5.5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-09-22.

Claude Code Containers DevOps and cloud Scientific agentic-tools ai-agents anthropic claude claude-skills

data-warehouse-experimentation

Running experiments out of the data warehouse instead of via dedicated experiment platforms. SQL-based assignment, exposure logging discipline, metric definitions in dbt models, statistical analysis in SQL or Python, variance reduction with CUPED, sequential testing, and the operational tradeoffs vs platforms like…

rampstackco/claude-skills · 157 tokens

data-scientist

Processes and analyzes data with resident-kernel engines (DuckDB, Polars) and one-shot tools. Use for CSV/parquet/JSON analysis, group-by/join/aggregation, time series, distributions, cleaning, or plotting a dataset.

code-yeongyu/oh-my-openagent · 54 tokens

agents-v2-py

Build container-based Foundry Agents with Azure AI Projects SDK (ImageBasedHostedAgentDefinition). Use when creating hosted agents with custom container images in Azure AI Foundry.

lingxling/awesome-skills-cn · 39 tokens

fortran

Best practices for modern Fortran (2003/2008+) scientific and numerical computing, covering modules, explicit interfaces, kind parameters, memory safety, and testing. Use when writing or reviewing Fortran source (.f90/.f95/.f03/.f08), defining modules and derived types, choosing numeric kind parameters, working with…

Mindrally/skills · 95 tokens

docker-debugger

Debug Docker containers, fix Dockerfile issues, optimize images, and troubleshoot docker-compose. Use when having Docker problems, container issues, or optimizing Docker builds.

OneWave-AI/claude-skills · 35 tokens

julia

Julia development guidelines covering multiple dispatch, type system, performance optimization, and scientific computing best practices.

Mindrally/skills · 22 tokens