Getting it into your agent
One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.
npx skills add aiopshwang/data-analysis-ml-agent-skills --skill designing-leakage-safe-experimentsgit clone --depth 1 https://github.com/aiopshwang/data-analysis-ml-agent-skillsWrote this? Show the measurements
A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.
[](https://agentmods.dev/skills/aiopshwang/data-analysis-ml-agent-skills/designing-leakage-safe-experiments)<a href="https://agentmods.dev/skills/aiopshwang/data-analysis-ml-agent-skills/designing-leakage-safe-experiments"><img src="https://agentmods.dev/badge/skills/aiopshwang/data-analysis-ml-agent-skills/designing-leakage-safe-experiments/github.svg" alt="Measured on agentmods" height="20"></a>Or the 80×15 button, for a site that already has a row of RSS and ATOM ones. Only the verdict fits; the numbers stay here.
<a href="https://agentmods.dev/skills/aiopshwang/data-analysis-ml-agent-skills/designing-leakage-safe-experiments"><img src="https://agentmods.dev/badge/skills/aiopshwang/data-analysis-ml-agent-skills/designing-leakage-safe-experiments.svg" alt="Reviewed on agentmods" width="80" height="20"></a>What it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5.1 | $0.00059 | $0.00728 |
| Opus 5 | $0.00030 | $0.00364 |
| Sonnet 5 | $0.00012 | $0.00146 |
| Haiku 4.5 | $0.00006 | $0.00073 |
Grade A, and why
designing-leakage-safe-experiments scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 11d ago.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Nothing flagged
None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.
How it starts
The opening of the file, as written. The whole thing — 56 lines — stays where its author put it; the contents beside it link to each section on GitHub.
Design Leakage-Safe Experiments
Create an experiment whose result would remain credible when the model meets new data in its intended operating environment. Leakage prevention starts with time and causality, not with a random split parameter.
Define the prediction event
Write down:
- entity and prediction grain;
- decision made from the prediction;
- prediction timestamp and feature cutoff;
- outcome definition and maturity window;
- eligible population at prediction time;
- deployment cadence, horizon, and retraining policy;
- operational cost of each error type.
Build an availability ledger for candidate features: source event, observation time, recording delay, correction or backfill behavior, earliest production availability, and whether the value depends on the outcome. Exclude or reconstruct features that would not exist in the same form at prediction time.
Read references/leakage-threats-and-splits.md when choosing a split or investigating a suspected leakage path.
Lock the comparison protocol
- Choose a split that matches deployment: temporal for future generalization, grouped for repeated entities, spatial or site holdout for new locations, or a justified combination.
- Reserve the final test set before feature selection or tuning. Repeated test inspection turns it into validation data.
- Fit preprocessing, imputation, encoding, scaling, selection, and resampling only on the training portion of each fold.
- Fit a calibrator from a separate calibration split or from out-of-fold predictions that did not train the base prediction for those records.
- Choose the operating threshold or decision policy from validation or calibration evidence. Keep the final test untouched until the model, calibrator, and policy are locked.
- Define primary and guardrail metrics before seeing candidate results.
- Establish a transparent baseline: prevalence, naive forecast, rule, or simple regularized model as appropriate.
- Hold data, split, metric implementation, and compute budget constant while comparing one material change at a time.
- Record every run, including failures and negative results. Complexity earns adoption only through a repeatable, decision-relevant improvement.
What ships with it
3 files beside SKILL.md in the same directory: the scripts, references and assets a skill reads on demand. Not counted in the per-session cost; read them before you install if any of them is executable.
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- 11d ago First seen · 56 lines · 59 tokens per session scan A fcfe90e90f4d
designing-leakage-safe-experiments is a skill published in the GitHub repository aiopshwang/data-analysis-ml-agent-skills (12 stars, last pushed 14d ago), licensed MIT. It adds 59 tokens to every session and 728 once invoked, about $0.0003 per session on Opus 5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-31.
Other skills, from other repositories
eu-ai-act-readiness
Build a preliminary, evidence-based EU AI Act readiness assessment across AI-system inventory, territorial scope, operator roles, prohibited-practice screening, risk classification, transparency, high-risk controls, general-purpose AI obligations, governance, and implementation milestones. Use when an organization…
data-labeling
Set up and manage data labeling workflows using manual annotation tools, semi-automated pipelines, active learning, and programmatic weak supervision. Use when the user requests data labeling or provides relevant inputs for this workflow.
knowledge-graph-creation
Build structured knowledge graphs from unstructured text by extracting entities, mapping relationships, generating graph triples, and visualizing the result. Use when the user requests knowledge graph creation or provides relevant inputs for this workflow.
literature-review
Conduct a structured literature review on a given topic by defining a search strategy, applying inclusion and exclusion criteria, extracting key findings, and synthesizing results into a coherent academic review. Use when the user requests literature review or provides relevant inputs for this workflow.
hyperparameter-tuning
Optimize machine learning model hyperparameters using grid search, random search, Bayesian optimization, and Hyperband to maximize model performance within a compute budget. Use when the user requests hyperparameter tuning or provides relevant inputs for this workflow.
model-training
Train machine learning models end-to-end, covering data loading, preprocessing, architecture selection, training loops, validation, and checkpointing. Use when the user requests model training or provides relevant inputs for this workflow.