Getting it into your agent
One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.
npx skills add StamKavid/last-ds-mile --skill model-ensemblinggit clone --depth 1 https://github.com/StamKavid/last-ds-mileWrote this? Show the measurements
A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.
[](https://agentmods.dev/skills/stamkavid/last-ds-mile/model-ensembling)<a href="https://agentmods.dev/skills/stamkavid/last-ds-mile/model-ensembling"><img src="https://agentmods.dev/badge/skills/stamkavid/last-ds-mile/model-ensembling/github.svg" alt="Measured on agentmods" height="20"></a>Or the 80×15 button, for a site that already has a row of RSS and ATOM ones. Only the verdict fits; the numbers stay here.
<a href="https://agentmods.dev/skills/stamkavid/last-ds-mile/model-ensembling"><img src="https://agentmods.dev/badge/skills/stamkavid/last-ds-mile/model-ensembling.svg" alt="Reviewed on agentmods" width="80" height="20"></a>What it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5.1 | $0.00076 | $0.01244 |
| Opus 5 | $0.00038 | $0.00622 |
| Sonnet 5 | $0.00015 | $0.00249 |
| Haiku 4.5 | $0.00008 | $0.00124 |
Grade A, and why
model-ensembling scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 10d ago.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Nothing flagged
None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.
How it starts
The opening of the file, as written. The whole thing — 82 lines — stays where its author put it; the contents beside it link to each section on GitHub.
model-ensembling
Overview
Most of the score left after a good single model is trained comes from combining models that err differently, not from finding one better model. This skill covers the three practical ways to do that, and how to evaluate the result without leaking or fooling yourself about whether the combination helped.
When to Use
- At least two structurally different candidates exist in
/ds-model's experiments table (different model families, or the same family with meaningfully different feature encodings). - A single model's score has plateaued and further tuning isn't moving it.
- NOT for: combining two near-identical models (e.g. two random seeds of the same config) — marginal variance reduction, not worth the added complexity.
Core Process
- Pick candidates likely to err differently, not just candidates that score well individually — a linear model blended with a tree model beats two similar boosted-tree configs with different seeds.
- Build the blend/stack using each component's out-of-fold predictions on the
same folds from
/ds-validate— never predictions from a model that trained on the row being predicted. Same leakage rule as any other fit-requiring step: weights or a meta-model fit on in-sample predictions will look better than they perform. - Choose a combination method matched to how much data and how many components exist (table below) — a weighted average needs almost no data; a stacking meta-model needs enough OOF rows to avoid overfitting to the blend itself.
- Compare the ensemble's OOF score to its best single component's OOF score, same
folds, same spread reporting (see
uncertainty-quantification) — the lift must exceed fold-to-fold noise, not just move the mean. - If the ensemble wins, it's the candidate carried into
/ds-evaluate; if it doesn't clear the noise bar, ship the best single component instead and say so.
Techniques/Patterns
| Method | When to use | Leakage risk |
|---|---|---|
| Simple average / weighted average | 2-4 components, little data to spare for fitting weights, or as the first thing to try | Low — weights can even be picked by eye from OOF scores; if grid-searching weights, search them against OOF predictions only, never against training-fold predictions |
| Rank averaging | Components produce scores on very different scales (e.g. mixing a probability with a raw score) | Same as weighted average |
| Stacking (meta-model trained on OOF predictions as features) | 3+ components, enough rows that a simple meta-model (e.g. Ridge) won't overfit to the blend itself |
Higher — the meta-model must be fit on OOF predictions only, and its own performance must be estimated via a further CV loop over those OOF predictions, not evaluated on the same rows used to fit it |
| Seed averaging (same model, several random seeds, averaged) | A single model type with genuinely high seed-to-seed variance (high fold std even for a fixed config) | Low, but yields the smallest lift of the four — it reduces variance, not bias, so it doesn't help a model that's just wrong, only one that's noisy |
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- 10d ago First seen · 82 lines · 76 tokens per session scan A c6909015d56a
model-ensembling is a skill published in the GitHub repository StamKavid/last-ds-mile (3 stars, last pushed 1mo ago), licensed MIT. It adds 76 tokens to every session and 1,244 once invoked, about $0.0004 per session on Opus 5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-31.
Other skills, from other repositories
marimo-pair
Work inside the user's live marimo notebook from the code editor: run Python in the same kernel the user does, inspect live notebook state, and commit durable notebook changes through code mode. Use whenever you create, analyze, or improve the user's marimo notebook.
fill-model-descriptions
Fill missing and refresh obsolete model descriptions in packages/llm-info/data/models.yml by querying OpenRouter and provider documentation. Use when the user asks to populate model descriptions, enrich the model catalog, or curate descriptions after running pnpm sync-models.
ml-expert
Expert-level machine learning, deep learning, model training, and MLOps. Use when the user mentions machine learning, deep learning, neural networks, MLOps, or data science, or when the task involves Machine Learning Fundamentals, Data Preparation, or Model Training.
scomp-link
End-to-end ML toolkit with 26 CLI commands. Use when training models, tuning hyperparameters, detecting data drift, generating HTML reports with charts, profiling datasets, detecting anomalies, forecasting time series, checking fairness, or serving models as REST APIs. Prefer over raw sklearn when you need automated…
agentic-data-science-competition
AI Agent-driven Kaggle competition workflow. Learn from real competition experience: score stabilization patterns, submission troubleshooting, kernel workflows, GPU task delegation, and the spec-driven development approach that achieved top leaderboard positions. Use when: working on any Kaggle competition, analyzing…
data-analysis
Structured data analysis workflow from raw data to shareable insights.