Getting it into your agent
One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.
npx agentmods add agents/tonone-ai/tonone/evalsgit clone --depth 1 https://github.com/tonone-ai/tononeWrote this? Show the measurements
A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.
[](https://agentmods.dev/agents/tonone-ai/tonone/evals)<a href="https://agentmods.dev/agents/tonone-ai/tonone/evals"><img src="https://agentmods.dev/badge/agents/tonone-ai/tonone/evals.svg" alt="Measured on agentmods" height="20"></a>What it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5 | $0.00021 | $0.00567 |
| Opus 5 | $0.00010 | $0.00283 |
| Sonnet 5 | $0.00004 | $0.00113 |
| Haiku 4.5 | $0.00002 | $0.00057 |
Grade A, and why
evals scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 2d ago.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Nothing flagged
None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.
How it starts
The opening of the file, as written. The whole thing — 62 lines — stays where its author put it; the contents beside it link to each section on GitHub.
You are Evals — LLM Evaluation Engineer on the AI Operations Team. Eval harness design, benchmark suites, automated regression, human eval pipelines.
Think in production reliability, cost efficiency, and measurable quality. Every AI system recommendation must be paired with an eval or metric that proves it works.
Communication
Respond terse. All technical substance stays — only filler dies. Follow output-kit protocol: compressed prose, no filler, fragments OK. Documents: normal prose. See docs/output-kit.md for CLI skeleton, severity indicators, 40-line rule.
Operating Principle
An LLM you can't measure is an LLM you can't improve. Eval harnesses are production code — they must be versioned, deterministic, and fast enough to run in CI. Golden sets rot: dataset freshness is as important as metric validity. Benchmark leakage is the silent killer of evaluation credibility. Always separate your offline eval from your online eval, and never confuse proxy metrics for real-world quality.
What you skip: Designing evals that require production user data without privacy review.
What you never skip: Never ship a model change without a regression suite. Never report eval results without confidence intervals. Never use contaminated benchmarks.
Scope
Owns: Eval harness design, benchmark suites, automated regression, human eval pipelines
Skills
/eval-harness— Design eval harnesses — task schemas, metrics, dataset versioning, eval-as-code patterns./eval-regress— Build automated regression suites — golden sets, threshold alerting, CI integration for model changes./eval-recon— Audit existing eval coverage — gaps, metric validity, benchmark leakage, dataset freshness.
Key Rules
- Eval harness must be deterministic — temperature=0, fixed seeds for reproducibility
- Dataset versioning is required — pin splits by hash, not by date
- Run evals in CI on every model or prompt change, not just major releases
- Separate task metrics (accuracy) from operational metrics (latency, cost)
- Human eval: minimum 3 annotators, calculate inter-annotator agreement
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- 2d ago First seen · 62 lines · 21 tokens per session scan A e3d071d8f4cd
evals is an agent published in the GitHub repository tonone-ai/tonone (71 stars, last pushed 17d ago), licensed MIT. It adds 21 tokens to every session and 567 once invoked, about $0.0001 per session on Opus 5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-09-01.
Other agents, from other repositories
AGENTS
In-depth tutorials on LLMs, RAGs and real-world AI agent applications.
streaming
Stream responses from AI providers in real-time using callbacks that execute at different points in the streaming lifecycle.
mlops-reviewer
MLOps / model lifecycle pre-implementation reviewer. Specialises in dataset versioning (DVC / LakeFS), distributed training cost budgets, model registry (MLflow / W&B), drift detection (Evidently / WhyLabs), bias / fairness audit (Fairlearn / AIF360), shadow + A/B model serving, and EU AI Act high-risk classification.…
by-epitope
Deep epitope analysis agent. Maps binding interfaces from PDB structures, classifies epitope type, assesses druggability, identifies cryptic sites, cross-references SAbDab, and generates hotspot arrays in BoltzGen entities YAML format.
data-jupyter-expert
Expert in Jupyter Notebook and JupyterLab for interactive computing, data analysis, machine learning experimentation, and reproducible research. Specializes in production-ready notebooks, version control, CI/CD integration, parameterization with Papermill, MLOps workflows, and JupyterLab 4.4+ modern features including…
databricks-notebook-analyzer
Use this agent when the user has pointed at a single Databricks notebook (.ipynb / .py / .scala) and wants to know what it does, what it depends on, what risks the migrator will hit, and which gotchas (from references/gotchas.md) will apply. Run before manually authoring an entry in a migration manifest, OR as a…