Getting it into your agent
One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.
npx agentmods add skills/stamkavid/last-ds-mile/error-analysisnpx skills add StamKavid/last-ds-mile --skill error-analysisgit clone --depth 1 https://github.com/StamKavid/last-ds-mileWrote this? Show the measurements
A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.
[](https://agentmods.dev/skills/stamkavid/last-ds-mile/error-analysis)<a href="https://agentmods.dev/skills/stamkavid/last-ds-mile/error-analysis"><img src="https://agentmods.dev/badge/skills/stamkavid/last-ds-mile/error-analysis.svg" alt="Measured on agentmods" height="20"></a>What it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5.1 | $0.00066 | $0.00828 |
| Opus 5 | $0.00033 | $0.00414 |
| Sonnet 5 | $0.00013 | $0.00166 |
| Haiku 4.5 | $0.00007 | $0.00083 |
Grade A, and why
error-analysis scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 5d ago.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Nothing flagged
None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.
How it starts
The opening of the file, as written. The whole thing — 69 lines — stays where its author put it; the contents beside it link to each section on GitHub.
error-analysis
Overview
An aggregate metric answers "how good on average" — error analysis answers "good for
whom, and wrong how." This is the systematic version of what /ds-evaluate does
inline, useful any time a deeper failure-mode investigation is needed.
When to Use
- After a model's aggregate metric is known and acceptable, before trusting it broadly.
- Debugging a specific complaint ("the model is bad for X") by finding the actual pattern, not guessing.
- NOT for: picking the metric itself (see
metric-selection) — this skill assumes the metric is already chosen and asks where performance diverges from that metric's average.
Core Process
- Compute out-of-fold (not in-sample) predictions for every row — error analysis on training-set-fit predictions is optimistic and misleading.
- Slice error by every dimension the decision cares about: a segment, a time period, a feature range, a subgroup — not just one arbitrary cut.
- Pull the worst N individual mispredictions and look for a shared trait (not just eyeball them one at a time with no hypothesis).
- State the finding as a concrete, actionable pattern ("underperforms on X because Y"), not a vague "some errors exist."
Techniques/Patterns
| Technique | What it reveals |
|---|---|
| Slice metric by categorical subgroup (segment, region, neighborhood, product line) | Whether one subgroup is silently much worse than the aggregate suggests |
| Slice metric by a continuous feature's quantile bins (e.g. price quintile, age decile) | Whether performance degrades at one end of a range — often the extremes |
| Residual vs. predicted-value plot (or a groupby summary of the same) | Systematic over/under-prediction bias at specific value ranges, not just noise |
| Worst-N mispredictions table, sorted by absolute error | Concrete cases to trace back to a shared root cause (a data issue, an undermodeled feature, an edge case) |
| Residual vs. time (for any temporal data) | Whether the model is drifting/degrading as time moves away from the training window |
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- 5d ago First seen · 69 lines · 66 tokens per session scan A 3c7beeeac465
error-analysis is a skill published in the GitHub repository StamKavid/last-ds-mile (3 stars, last pushed 28d ago), licensed MIT. It adds 66 tokens to every session and 828 once invoked, about $0.0003 per session on Opus 5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-31.
Other skills, from other repositories
understanding-streamlit-architecture
Explains Streamlit's internal architecture including backend runtime, frontend rendering, and WebSocket communication. Use when debugging cross-layer issues, understanding how features work end-to-end, planning architectural changes, or onboarding to the codebase. Covers ForwardMsg/BackMsg protocol, script rerun…
fixing-streamlit-ci
Analyze and fix failed GitHub Actions CI jobs for the current branch/PR. Use when CI checks fail, PR checks show failures, or you need to diagnose lint/type/test errors and verify fixes locally.
updating-internal-docs
Review internal documentation (.md files) against the current codebase state and propose updates for outdated or incorrect information.
assessing-external-test-risk
Assesses whether branch or PR changes are high-risk for externally hosted or embedded Streamlit usage and recommends whether external e2e coverage with @pytest.mark.externaltest is needed. Use during code review, PR triage, or test planning when changes touch routing, auth, websocket/session behavior, embedding…
sharing-pr-agent-artifacts
Uploads agent-generated artifacts (specs, plans, learnings) to the streamlit.wiki for sharing via PR comments. Use when you have agent artifacts to share with reviewers.
developing-with-streamlit
Use for ALL Streamlit tasks: creating, editing, debugging, beautifying, styling, theming, optimizing, or deploying Streamlit apps. Also custom components, st.components.v2, HTML/JS/CSS work. Discovers and loads version-matched reference docs from the user's installed Streamlit (>=1.57). Triggers: streamlit, st.…