training-pipeline-debugging

training-pipeline-debugging is a skill for Claude Code from yeaight7/agent-powerups. It costs 42 tokens per session (528 once invoked), scanned A, original, Apache-2.0.

A troubleshooting guide for machine-learning training failures such as invalid losses, memory exhaustion, tensor shape errors, or a model that does not learn. It starts by testing whether the model can fit one small batch.

In plain words
What is it for?
Use it to investigate NaN losses, out-of-memory failures, dimension mismatches, exploding gradients, data problems, and training loops that fail to learn.
Why use it?
It narrows down silent mathematical and data problems before spending time on full training runs.

Skill for Claude Code

Written for Claude Code: shipped in a Claude Code plugin.

Part of the machine-learning-ops plugin — 7 skills, 4 commands, 3 agents shipped together

Good fit Use it to investigate NaN losses, out-of-memory failures, dimension mismatches, exploding gradients, data problems, and training loops that fail to learn.

Compare 6 skills from other repositories ↓
Install with agentmods
npx agentmods add skills/yeaight7/agent-powerups/training-pipeline-debugging
Install

Getting it into your agent

One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.

Any agent
npx skills add yeaight7/agent-powerups --skill training-pipeline-debugging
Clone the repo
git clone --depth 1 https://github.com/yeaight7/agent-powerups

Made for: Claude Code.

Or install machine-learning-ops, the plugin that ships this one along with the rest of its 7 skills, 4 commands, 3 agents.

Wrote this? Show the measurements

A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.

agentmods badge for training-pipeline-debugging

README.md
[![agentmods](https://agentmods.dev/badge/skills/yeaight7/agent-powerups/training-pipeline-debugging/github.svg)](https://agentmods.dev/skills/yeaight7/agent-powerups/training-pipeline-debugging)
Your own site
<a href="https://agentmods.dev/skills/yeaight7/agent-powerups/training-pipeline-debugging"><img src="https://agentmods.dev/badge/skills/yeaight7/agent-powerups/training-pipeline-debugging/github.svg" alt="Measured on agentmods" height="20"></a>

Or the 80×15 button, for a site that already has a row of RSS and ATOM ones. Only the verdict fits; the numbers stay here.

agentmods 80×15 button for training-pipeline-debugging

Your own site · 80×15
<a href="https://agentmods.dev/skills/yeaight7/agent-powerups/training-pipeline-debugging"><img src="https://agentmods.dev/badge/skills/yeaight7/agent-powerups/training-pipeline-debugging.svg" alt="Reviewed on agentmods" width="80" height="20"></a>
Per session 42 Skills are progressive disclosure: only the name and description are preloaded; the body loads when the skill is used.
When invoked 528 The whole file, excluding the scripts and references it only reads on demand.
Security scan A 0 findings. A grade says what 26 rules found in the file — not that it is safe.
Origin original No closer match found in the catalogue.
Token cost

What it costs to keep this loaded

Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.

ModelPer sessionOnce invoked
Fable 5.1 $0.00042 $0.00528
Opus 5.5 $0.00017 $0.00211
Sonnet 5.5 $0.00008 $0.00106
Haiku 4.5 $0.00004 $0.00053

Measured 1mo ago against content hash accac340a4a8, method: parsed. Prices are Anthropic first-party input rates as of 2026-10-07, from the pricing page.

Security

Grade A, and why

training-pipeline-debugging scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 1mo ago.

A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.

Nothing flagged

None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.

plugins/machine-learning-ops/skills/training-pipeline-debugging/SKILL.md · 53 lines

How it starts

The opening of the file, as written. The whole thing — 53 lines — stays where its author put it; the contents beside it link to each section on GitHub.

Purpose

ML training bugs are often silent mathematical errors rather than explicit code crashes. This protocol localizes the standard failure classes quickly.

When to Use

  • Loss becomes NaN or diverges
  • Training crashes with out-of-memory errors
  • Shape/dimension errors at matrix multiplications or loss computation
  • The pipeline runs but the model never learns

Inputs

  • The failing training script and its error output or loss curve

Workflow

  1. Run the overfit test first: fit a single batch of ~10 examples. If the model cannot achieve near-zero loss on a single batch, the pipeline is fundamentally broken — do not debug full runs until the single-batch test passes.
  2. NaN losses — check:
    • Learning rate too high?
    • Missing data (NaNs in input)?
    • Log/Exp/Divide-by-zero in custom loss functions?
    • Exploding gradients (clip gradients)?
  3. OOM (out of memory):
    • Reduce batch size.
    • Check for memory leaks in the training loop (e.g., accumulating history across epochs without .detach()).
  4. Shape mismatches:
    • Add temporary print statements or assertions on tensor.shape before matrix multiplications and loss calculations.

Output

  • The identified failure class and its specific fix, or a single-batch reproduction showing exactly where the pipeline breaks

Verification

  • Single-batch overfit test run before any full-run debugging
  • For NaN: learning rate, input NaNs, unsafe math, and gradient explosion all checked
  • For OOM: batch size and training-loop accumulation both checked
  • For shapes: assertions placed before matmul/loss sites
  • Fix validated by re-running the previously failing case

Failure Modes

  • Debugging full runs first — hours per iteration; the single-batch test gives answers in minutes.
  • Treating NaN as random — NaN losses have a small set of causes; check all four systematically instead of restarting with a lower LR.
  • Batch-size-only OOM fixes — shrinking the batch hides a loop leak that will OOM again later; check accumulation too.
  • Leftover instrumentation — remove temporary shape prints/assertions once the fix is validated.

Read the full file on GitHub · 53 lines

Changes

What this file has done since we first saw it

Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.

  1. 1mo ago First seen · 53 lines · 42 tokens per session scan A accac340a4a8

Subscribe to this mod's changes

training-pipeline-debugging is a skill published in the GitHub repository yeaight7/agent-powerups (6 stars, last pushed 19d ago), licensed Apache-2.0. It adds 42 tokens to every session and 528 once invoked, about $0.0002 per session on Opus 5.5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-09-14.