NVIDIA-NeMo/Gym is a library and infrastructure for evaluating and improving models and agents inside environments, where each environment defines tasks, agent interaction, verification, and execution state. It is for teams running reproducible evaluations or training at scale across settings such as code execution, tool calling, and sandboxes, and the catalogue entries provide skills and instructions for working with it.
Getting it into your agent
One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.
npx agentmods add skills/nvidia-nemo/gym/nemo-gym-debuggingnpx skills add NVIDIA-NeMo/Gym --skill nemo-gym-debugginggit clone --depth 1 https://github.com/NVIDIA-NeMo/GymWrote this? Show the measurements
A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.
[](https://agentmods.dev/skills/nvidia-nemo/gym/nemo-gym-debugging)<a href="https://agentmods.dev/skills/nvidia-nemo/gym/nemo-gym-debugging"><img src="https://agentmods.dev/badge/skills/nvidia-nemo/gym/nemo-gym-debugging.svg" alt="Measured on agentmods" height="20"></a>What it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5.1 | $0.00068 | $0.00785 |
| Opus 5 | $0.00034 | $0.00392 |
| Sonnet 5 | $0.00014 | $0.00157 |
| Haiku 4.5 | $0.00007 | $0.00078 |
Grade A, and why
nemo-gym-debugging scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 5d ago.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Nothing flagged
None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.
How it starts
The opening of the file, as written. The whole thing — 62 lines — stays where its author put it; the contents beside it link to each section on GitHub.
Nemo Gym Debugging
Invocation Check
Use this skill when something failed or looks suspicious in a Nemo Gym run. If the task is adding a new env, use the nemo-gym-env-integration skill; if it is changing profiling behavior, use the nemo-gym-reward-profiling skill.
Debug by classification, not by guessing. The first goal is to decide whether the issue is:
- infra: Slurm, Ray, container, filesystem, network, ports
- model serving: vLLM startup/readiness/throughput
- config: wrong config bundle, missing agent, wrong extra args
- data/schema: JSONL fields do not match verifier/resource server expectations
- verifier/runtime: resource server exception or malformed verify response
- cache/resume: stale materialized inputs or partial rollout output
- throughput/resources: concurrency too high, judge bottleneck, tool/sandbox latency
Debug Order
- Check Slurm/Ray job state and logs.
- Check vLLM readiness and
/modelsavailability. - Check Gym server readiness: all expected servers started.
- Check tool routing if the env uses tools; check sandbox readiness only if a sandbox is configured.
- Check materialized inputs and source data timestamps.
- Check rollout output and profiling/metrics output counts.
- Inspect the first real verifier exception, not shutdown noise.
- Compare failing row schema against the resource server request model.
High-Value Suspects
- If data changed and
resume_from_cachewas enabled, stale materialized inputs are a first-class suspect. - If rollout output has a few rows and profiling is empty, inspect verifier errors and partial-output cache.
- If all servers are ready but verifier returns 422/500, inspect request body schema before debugging infra.
- If tool envs hang or partially work, check tool ownership/loading before changing model settings; check sandbox readiness only when a sandbox is actually part of the env.
- If tool-call rows fail before generation with vLLM grammar/schema errors, read
references/vllm-tool-call-schema-checks.mdand run a static tool-schema check before changing Gym wrappers. - If logs only show nested "inner server" 500s without the real provider/verifier body, first enable existing request-boundary visibility with
++global_aiohttp_client_request_debug=True. Readreferences/request-boundary-visibility.mdbefore changing code.
What ships with it
6 files beside SKILL.md in the same directory: the scripts, references and assets a skill reads on demand. Not counted in the per-session cost; read them before you install if any of them is executable.
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- 5d ago First seen · 62 lines · 68 tokens per session scan A 1f005e915095
nemo-gym-debugging is a skill published in the GitHub repository NVIDIA-NeMo/Gym (1,164 stars, last pushed today), licensed Apache-2.0. It adds 68 tokens to every session and 785 once invoked, about $0.0003 per session on Opus 5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-30.
Other skills, from other repositories
writing-bench-task-judge
Use when writing or modifying checkgoals() / getanswer() / App check methods in benchenv/task/, or when reviewing a draft task's judge correctness. Triggers include adding a new task, editing a judge method, or diagnosing a judge false-positive/negative.
testing-bench-task
Use when adding or modifying offline judge tests for benchenv tasks — specifically entries in OFFLINEJUDGEPOSITIVECASES / OFFLINEJUDGENEGATIVECASES in benchenv/tests/ /testtasks.py, or writing live tests. Triggers after a new task is added, or when tightening judge coverage.
designing-bench-task
Use when designing a new benchenv task suite, adding several new tasks to an existing suite, or critiquing a task-set proposal for a mobile-gym App — before any class FooTask(...) is written under benchenv/task/.
model_finetuning
Fine-tune LLMs using reinforcement learning with TRL - SFT for instruction tuning, DPO for preference alignment, PPO/GRPO for reward optimization, and reward model training. Use when need RLHF, align model with preferences, or train from human feedback. Works with HuggingFace Transformers.
model_finetuning
Fine-tune LLMs using reinforcement learning with TRL - SFT for instruction tuning, DPO for preference alignment, PPO/GRPO for reward optimization, and reward model training. Use when need RLHF, align model with preferences, or train from human feedback. Works with HuggingFace Transformers.
foundry-config-setup
Resolve missing setup caused by a hardcoded Foundry project endpoint or model in a sample. Use when a sample fails because it uses a placeholder/hardcoded projectendpoint (for example "https://your-project.services.ai.azure.com") or a hardcoded model instead of reading them from the environment.