Borrowing it
Nothing to install: this file belongs to stellar-experimental/stellar-raven. Take a copy, put it at the same path in your own repository, and replace the rules that are about this project with yours.
curl -O https://raw.githubusercontent.com/stellar-experimental/stellar-raven/main/.agents/skills/run-evals/SKILL.mdgit clone --depth 1 https://github.com/stellar-experimental/stellar-ravenWrote this? Show the measurements
A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.
[](https://agentmods.dev/skills/stellar-experimental/stellar-raven/run-evals)<a href="https://agentmods.dev/skills/stellar-experimental/stellar-raven/run-evals"><img src="https://agentmods.dev/badge/skills/stellar-experimental/stellar-raven/run-evals/github.svg" alt="Measured on agentmods" height="20"></a>Or the 80×15 button, for a site that already has a row of RSS and ATOM ones. Only the verdict fits; the numbers stay here.
<a href="https://agentmods.dev/skills/stellar-experimental/stellar-raven/run-evals"><img src="https://agentmods.dev/badge/skills/stellar-experimental/stellar-raven/run-evals.svg" alt="Reviewed on agentmods" width="80" height="20"></a>- NVIDIA SkillSpector warn
SkillSpector: 2 findings, up to medium
These are SkillSpector’s own severities. On a checked sample its high-severity flags on skills were ~96% false positives — a documented command, a public API, a “never do X” rule — so we show them as a caution to read, not a verdict. Why →
- medium Data Exfiltration · line 258 Data is being sent to an external URL. This could be legitimate telemetry or data exfiltration. Manual review is recommended.Fix: Verify the destination URL is trusted and necessary. Remove or replace with documented APIs. Ensure no secrets, tokens, or PII are transmitted.
- medium Agent Snooping · line 576 Skill enumerates or reads other installed skills. Access to other skills' SKILL.md files or the skills directory reveals prompt instructions, capabilities, and secrets that should be invisible to peer skills.Fix: Remove all code or instructions that list or read other skills' files or directories. Skills should operate independently; cross-skill access is a privilege escalation.
What it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5.1 | $0.00137 | $0.07848 |
| Opus 5.5 | $0.00055 | $0.03139 |
| Sonnet 5.5 | $0.00027 | $0.01570 |
| Haiku 4.5 | $0.00014 | $0.00785 |
Grade A, and why
run-evals scanned grade A with 1 finding against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 28d ago.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Makes network callslowCapability
Not a fault in itself. Listed so you know the mod talks to something, and to what.
curl -s -o /dev/null -w '%{http_code}' -X POST "http://localhost:${PORT}/mcp" \ How it starts
The opening of the file, as written. The whole thing — 487 lines — stays where its author put it; the contents beside it link to each section on GitHub.
Eval round — stellar-raven-codemode
This skill is the orchestration around the instruments. The instruments and their contracts live
in eval/EVALS.md, eval/README.md, and eval/qa/README.md. Re-read eval/EVALS.md and
improvements/README.md before any round; they are the current truth.
North star
Every instrument answers one question at some layer:
Does an agent driving this MCP end-to-end produce a correct, current, non-fabricated answer?
Two non-negotiables:
- The scores are the instrument; the findings are the product. This server's own tuning
ceiling is single-digit points. The outsized leverage is discovering gaps in the four
upstream surfaces (Lumenloop, Stellar Light/Scout, Stellar Docs, skill sources). A round
that surfaces an upstream gap and doesn't file it in
improvements/has dropped its most valuable output. - One headline, three routing gates, everything else diagnostic. The headline is QA
end-to-end (contracts in
eval/qa/README.md). The gates are the legacy, skills, and holdout routing lanes (thresholds ineval/gates.json). Never merge lanes, never tune per-question, never promote a view to a gate without a decision recorded in the round ledger.
Agent roles and model boundaries
Do not conflate the agent running this runbook with the model under test:
- Orchestrating agent: Codex/Claude Code/etc. in the repo. It starts servers, runs commands, records result stamps, joins rows with goldens, reviews transcripts, patches code/docs, and files findings. It is not the QA answer model being measured.
- Answering agent: spawned by
eval/qa/run-qa.mjsonce per QA case via headlessclaude -p. It only gets the MCPsearch+executetools and produces the candidate user-facing answer. Default model:claude-sonnet-5, override with--model. - Judge agent: spawned by
eval/qa/judge.mjsto grade the candidate answer against the golden. Default model:claude-sonnet-5, override with--judge-model. Judge verdicts are evidence to review, not unquestionable truth.
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- 28d ago Changed · -234 lines 01bfddc3c24a
- 29d ago Changed · +1 lines 29951ce1f9ed
- 1mo ago Changed · -5 lines 2c55d7751fc4
- 1mo ago Changed · +47 lines 1ed82e192eeb
- 1mo ago Changed · +19 lines 47b079032022
- 1mo ago First seen · 659 lines · 137 tokens per session scan A a097f22c92f6
run-evals is a skill published in the GitHub repository stellar-experimental/stellar-raven (13 stars, last pushed yesterday), licensed Apache-2.0. It adds 137 tokens to every session and 7,848 once invoked, about $0.0005 per session on Opus 5.5. A static security scan graded it A with 1 finding (makes network calls). No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-31.
Other skills, from other repositories
foundry-hosted-agent-validation
Step-by-step process for validating a Python Foundry hosted agent sample (under python/samples/04-hosting/foundry-hosted-agents/) end to end — running it locally (native runtime and azd ai agent run) and after deploying it to an Azure AI Foundry project with azd. Use this when asked to validate a hosted agent sample.
build-and-test
How to build and test .NET projects in the Agent Framework repository. Use this when verifying or testing changes.
skill-release-gate
Evaluate an Agent Skill bundle for structural integrity, trigger quality, artifact improvement, script correctness, safety, installed-tree integrity, and target-host portability before release.
verify-omnigent
Drive Omnigent the way a user does and prove a behavior with recorded evidence, using an isolated server, runner, and mock model plus a feature map of every user entry point. Load before reproducing a user-facing bug, before claiming a fix works, or when reviewing whether a change covered every surface (session vs.…
adk-go-self-review
Review an ADK Go change the way a maintainer will — a fresh-context pass over the whole diff, five lenses (correctness and tests, scope, simplicity, style, adk-python parity), and the mutation check that proves your tests pin the change. Use before opening a PR, before any later push that changes code, and when asked…
python-sdk
Implement or modify Python SDK behavior under python/composio, including tools, toolkits, sessions, auth configs, connected accounts, client integration, and shared Python models. Use for Python core runtime/API work; pair with python-testing and cross-sdk-parity when TypeScript must match.