Getting it into your agent
One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.
npx agentmods add skills/hoja-solutions/agent-stdlib/calibrate-eval-infrastructurenpx skills add Hoja-Solutions/agent-stdlib --skill calibrate-eval-infrastructuregit clone --depth 1 https://github.com/Hoja-Solutions/agent-stdlibWrote this? Show the measurements
A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.
[](https://agentmods.dev/skills/hoja-solutions/agent-stdlib/calibrate-eval-infrastructure)<a href="https://agentmods.dev/skills/hoja-solutions/agent-stdlib/calibrate-eval-infrastructure"><img src="https://agentmods.dev/badge/skills/hoja-solutions/agent-stdlib/calibrate-eval-infrastructure.svg" alt="Measured on agentmods" height="20"></a>What it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5 | $0.00147 | $0.00828 |
| Opus 5 | $0.00073 | $0.00414 |
| Sonnet 5 | $0.00029 | $0.00166 |
| Haiku 4.5 | $0.00015 | $0.00083 |
Grade A, and why
calibrate-eval-infrastructure scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 4d ago.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Nothing flagged
None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.
How it starts
The opening of the file, as written. The whole thing — 66 lines — stays where its author put it; the contents beside it link to each section on GitHub.
Calibrate eval infrastructure
Source: Quantifying infrastructure noise in agentic coding evals. This had no packaged skill anywhere; the topic existed only as the article and a few summaries.
Container configuration can move an agentic coding benchmark by 6 or more points. That is larger than the gap between the top models, which means a careless resource setting can rank a worse model above a better one. Treat resource configuration as an experimental variable you control and report, on the same footing as prompt format and sampling temperature.
The mistake that causes most of it
A container has two separate numbers, and pinning them together is the trap:
- a guaranteed allocation, the floor the workload always has, and
- a hard kill threshold, the ceiling past which the runtime kills the process.
Set the floor equal to the ceiling and the workload has zero headroom. A normal memory spike crosses the line and the process dies an OOM death that looks like the agent failing the task. It was not the agent. It was the box.
Give the two numbers separate values and leave a band between them.
Size the band empirically
Do not guess the headroom. Sweep it and measure whether the score still moves:
- Run the benchmark at several ceilings (a useful starting reference is roughly 3x the baseline ceiling; Anthropic's sweep cut infra errors from 5.8% to 2.1% with negligible score change at that point).
- Run each configuration several times, because one run per config cannot separate signal from noise.
- Test whether the score differences across configurations are distinguishable from noise. If they are not, you have found a band wide enough that the infrastructure no longer decides the outcome.
The bundled script runs that test:
python scripts/noise_check.py --scores scores.json
It reports a permutation p-value across your configs. A high p-value (the article treats roughly 0.40 and above as within noise) means resource configuration is no longer moving the score, so any model differences you measure are real. A low p-value means the infrastructure is still talking, and you need a wider band before you trust a single number. See scripts/noise_check.py for the input format.
What ships with it
1 file beside SKILL.md in the same directory: the scripts, references and assets a skill reads on demand. Not counted in the per-session cost; read them before you install if any of them is executable.
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- 4d ago First seen · 66 lines · 147 tokens per session scan A 88b944d7ac02
calibrate-eval-infrastructure is a skill published in the GitHub repository Hoja-Solutions/agent-stdlib (1 stars, last pushed 1mo ago), licensed MIT. It adds 147 tokens to every session and 828 once invoked, about $0.0007 per session on Opus 5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-31.
Other skills, from other repositories
agent-sandbox
Agent skill for sandbox - invoke with $agent-sandbox.
github-actions-templates
Create production-ready GitHub Actions workflows for automated testing, building, and deploying applications. Use when setting up CI/CD with GitHub Actions, automating development workflows, or creating reusable workflow templates.
aiq-deploy
Use when asked to install, deploy, run, validate, troubleshoot, or stop NVIDIA AI-Q Blueprint infrastructure.
dynamo-recipe-runner
Select, validate, patch, and deploy existing NVIDIA Dynamo Kubernetes recipes. Use for model/backend/GPU/deployment-mode recipe bring-up; use router-starter for router-only mode work and troubleshoot for broken deployments.
holoscan-install-container
Install Holoscan SDK via the NGC Docker container. Use for container-based installs; not for native apt/pip/Conda installs.
doca-argus
Use this skill when the user is deploying or operating the DOCA Argus Service — the packaged BlueField-side runtime-security container that watches the BlueField and attached host for suspicious activity, integrity violations, and operational anomalies, and forwards findings to a SIEM (Splunk / ELK / Sentinel /…