Getting it into your agent
One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.
npx skills add yeaight7/agent-powerups --skill ml-leakage-checkgit clone --depth 1 https://github.com/yeaight7/agent-powerupsWrote this? Show the measurements
A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.
[](https://agentmods.dev/skills/yeaight7/agent-powerups/ml-leakage-check)<a href="https://agentmods.dev/skills/yeaight7/agent-powerups/ml-leakage-check"><img src="https://agentmods.dev/badge/skills/yeaight7/agent-powerups/ml-leakage-check/github.svg" alt="Measured on agentmods" height="20"></a>Or the 80×15 button, for a site that already has a row of RSS and ATOM ones. Only the verdict fits; the numbers stay here.
<a href="https://agentmods.dev/skills/yeaight7/agent-powerups/ml-leakage-check"><img src="https://agentmods.dev/badge/skills/yeaight7/agent-powerups/ml-leakage-check.svg" alt="Reviewed on agentmods" width="80" height="20"></a>What it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5.1 | $0.00040 | $0.00439 |
| Opus 5.5 | $0.00016 | $0.00176 |
| Sonnet 5.5 | $0.00008 | $0.00088 |
| Haiku 4.5 | $0.00004 | $0.00044 |
Grade A, and why
ml-leakage-check scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 1mo ago.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Nothing flagged
None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.
What it actually says
Purpose
Target leakage is the most common and dangerous error in applied ML. It creates models that look perfect in validation but fail instantly in production. This check inspects the pipeline for the standard leakage vectors.
When to Use
- Reviewing preprocessing or feature-engineering code before training sign-off
- Validation metrics look too good to be true
- A model performed far worse in production than in validation
Inputs
- The preprocessing/feature pipeline code and the train/test split logic
Workflow
- Global scaling/imputation: was any statistic (mean, std, encoder vocabulary) computed on the entire dataset before splitting? That leaks the test distribution into training.
- Future features: is any training feature unavailable at the moment of prediction in real life? (e.g., using "surgery_outcome" to predict "hospital_admission_length")
- ID proxies: are database IDs or row numbers included as features? They often correlate with time or order of entry.
- Enforce the order: Split FIRST, then fit transformers on Train ONLY, then transform Train/Val/Test.
Output
- A leakage verdict per vector (global statistics, future features, ID proxies), citing the offending lines and the fix for each
Verification
- All fit/fit_transform calls occur after the split and only on training data
- Every feature audited for availability at prediction time
- No raw IDs or row numbers among the features
- Findings cite specific code lines
Failure Modes
- Pipeline blindness — leakage hidden inside helper functions or composed pipelines; trace where fitting actually happens.
- "It's just scaling" — dismissing global scaling as harmless; it still leaks distribution information.
- Fixing one vector only — re-running after one fix while the other vectors remain unchecked; audit all three every time.
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- 1mo ago First seen · 43 lines · 40 tokens per session scan A 20deb9f3b8db
ml-leakage-check is a skill published in the GitHub repository yeaight7/agent-powerups (6 stars, last pushed 19d ago), licensed Apache-2.0. It adds 40 tokens to every session and 439 once invoked, about $0.0002 per session on Opus 5.5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-09-14.
Other skills, from other repositories
systematic-debugging
4-phase root cause debugging: understand bugs before fixing.
simplify-code
Sequential 3-lens cleanup of recent code changes.
node-inspect-debugger
Debug Node.js via --inspect + Chrome DevTools Protocol.
python-debugpy
Debug Python: pdb REPL + debugpy remote (DAP).
hqe
Comprehensive codebase health auditing, remediation, and verification skill based on the canonical HQE Protocol v5.0.0.
bigquery-ai-ml
Leverages BigQuery's built-in machine learning and GenAI capabilities for advanced data analytics. Use when you need to write SQL queries that perform time-series forecasting, predict values, detect outliers or anomalies, find key drivers, perform semantic search or vector search, classify text, calculate similarity…