Getting it into your agent
One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.
npx skills add aiopshwang/data-analysis-ml-agent-skills --skill diagnosing-ml-failuresgit clone --depth 1 https://github.com/aiopshwang/data-analysis-ml-agent-skillsWrote this? Show the measurements
A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.
[](https://agentmods.dev/skills/aiopshwang/data-analysis-ml-agent-skills/diagnosing-ml-failures)<a href="https://agentmods.dev/skills/aiopshwang/data-analysis-ml-agent-skills/diagnosing-ml-failures"><img src="https://agentmods.dev/badge/skills/aiopshwang/data-analysis-ml-agent-skills/diagnosing-ml-failures/github.svg" alt="Measured on agentmods" height="20"></a>Or the 80×15 button, for a site that already has a row of RSS and ATOM ones. Only the verdict fits; the numbers stay here.
<a href="https://agentmods.dev/skills/aiopshwang/data-analysis-ml-agent-skills/diagnosing-ml-failures"><img src="https://agentmods.dev/badge/skills/aiopshwang/data-analysis-ml-agent-skills/diagnosing-ml-failures.svg" alt="Reviewed on agentmods" width="80" height="20"></a>What it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5.1 | $0.00065 | $0.00832 |
| Opus 5 | $0.00032 | $0.00416 |
| Sonnet 5 | $0.00013 | $0.00166 |
| Haiku 4.5 | $0.00006 | $0.00083 |
Grade A, and why
diagnosing-ml-failures scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 12d ago.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Nothing flagged
None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.
How it starts
The opening of the file, as written. The whole thing — 59 lines — stays where its author put it; the contents beside it link to each section on GitHub.
Diagnosing ML Failures
Turn a vague failure into a minimal, testable discrepancy and isolate the first layer where expected and observed behavior diverge. Do not begin with hyperparameter tuning: it can hide the fault while destroying causal evidence.
Define the symptom
Capture one concrete failing example or comparison:
- expected versus observed behavior;
- first known bad and last known good run, version, or time window;
- affected population and unaffected control population;
- metric implementation, threshold, and aggregation level;
- exact input, artifact, code revision, configuration, and environment when available.
Reproduce the symptom through the smallest stable entry point. If it is intermittent, estimate frequency and identify what varies between runs before changing the system.
Isolate the first broken layer
Trace the same records through boundaries in causal order:
- Source and semantics: extraction window, identifiers, units, timestamps, schema, missingness, duplicates, and joins.
- Ground truth: label definition, observation window, reviewer process, label delay, drift, and adjudication.
- Split and sampling: entity overlap, temporal leakage, cohort shift, weighting, and deduplication.
- Feature pipeline: availability time, fit/transform state, ordering, encoding, defaults, and training-serving parity.
- Model artifact: weights, feature contract, class order, preprocessing bundle, and artifact version.
- Metric and decision logic: denominator, grouping, threshold, calibration, exclusions, and business-cost mapping.
- Runtime and delivery: dependency versions, hardware nondeterminism, caching, serialization, concurrency, and response postprocessing.
Compare invariants at each boundary - row counts, key uniqueness, hashes, distributions, schemas, representative record traces - and stop at the earliest divergence. Downstream differences may be consequences, not causes.
Test competing hypotheses
Maintain multiple plausible hypotheses until evidence distinguishes them. For each hypothesis, define a discriminating test whose outcomes differ between explanations. Prefer controlled swaps and minimal counterfactuals:
What ships with it
3 files beside SKILL.md in the same directory: the scripts, references and assets a skill reads on demand. Not counted in the per-session cost; read them before you install if any of them is executable.
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- 12d ago First seen · 59 lines · 65 tokens per session scan A f802c3724023
diagnosing-ml-failures is a skill published in the GitHub repository aiopshwang/data-analysis-ml-agent-skills (12 stars, last pushed 16d ago), licensed MIT. It adds 65 tokens to every session and 832 once invoked, about $0.0003 per session on Opus 5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-31.
Other skills, from other repositories
eu-ai-act-readiness
Build a preliminary, evidence-based EU AI Act readiness assessment across AI-system inventory, territorial scope, operator roles, prohibited-practice screening, risk classification, transparency, high-risk controls, general-purpose AI obligations, governance, and implementation milestones. Use when an organization…
data-labeling
Set up and manage data labeling workflows using manual annotation tools, semi-automated pipelines, active learning, and programmatic weak supervision. Use when the user requests data labeling or provides relevant inputs for this workflow.
model-training
Train machine learning models end-to-end, covering data loading, preprocessing, architecture selection, training loops, validation, and checkpointing. Use when the user requests model training or provides relevant inputs for this workflow.
debugging
Systematically diagnose and fix software bugs by analyzing error messages, stack traces, logs, and runtime behavior across multiple languages. Use when the user requests debugging or provides relevant inputs for this workflow.
context-injection
Place trusted contextual information into prompts or agent state using explicit boundaries, provenance, and templates. Use when relevant context has already been selected and must be inserted safely; use context-retrieval to find it or context-optimization to choose and order it.
knowledge-graph-creation
Build structured knowledge graphs from unstructured text by extracting entities, mapping relationships, generating graph triples, and visualizing the result. Use when the user requests knowledge graph creation or provides relevant inputs for this workflow.