Borrowing it
Nothing to install: this file belongs to stellar-experimental/stellar-raven. Take a copy, put it at the same path in your own repository, and replace the rules that are about this project with yours.
curl -O https://raw.githubusercontent.com/stellar-experimental/stellar-raven/main/.agents/skills/run-evals/SKILL.mdgit clone --depth 1 https://github.com/stellar-experimental/stellar-ravenWrote this? Show the measurements
A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.
[](https://agentmods.dev/skills/stellar-experimental/stellar-raven/run-evals)<a href="https://agentmods.dev/skills/stellar-experimental/stellar-raven/run-evals"><img src="https://agentmods.dev/badge/skills/stellar-experimental/stellar-raven/run-evals.svg" alt="Measured on agentmods" height="20"></a>- NVIDIA SkillSpector warn
SkillSpector: 2 findings, up to medium
These are SkillSpector’s own severities. On a checked sample its high-severity flags on skills were ~96% false positives — a documented command, a public API, a “never do X” rule — so we show them as a caution to read, not a verdict. Why →
- medium Data Exfiltration · line 258 Data is being sent to an external URL. This could be legitimate telemetry or data exfiltration. Manual review is recommended.Fix: Verify the destination URL is trusted and necessary. Remove or replace with documented APIs. Ensure no secrets, tokens, or PII are transmitted.
- medium Agent Snooping · line 576 Skill enumerates or reads other installed skills. Access to other skills' SKILL.md files or the skills directory reveals prompt instructions, capabilities, and secrets that should be invisible to peer skills.Fix: Remove all code or instructions that list or read other skills' files or directories. Skills should operate independently; cross-skill access is a privilege escalation.
What it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5.1 | $0.00137 | $0.12101 |
| Opus 5 | $0.00068 | $0.06051 |
| Sonnet 5 | $0.00027 | $0.02420 |
| Haiku 4.5 | $0.00014 | $0.01210 |
Grade A, and why
run-evals scanned grade A with 1 finding against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 3d ago.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Makes network callslowCapability
Not a fault in itself. Listed so you know the mod talks to something, and to what.
curl -s -o /dev/null -w '%{http_code}' -X POST "http://localhost:${PORT}/mcp" \ How it starts
The opening of the file, as written. The whole thing — 725 lines — stays where its author put it; the contents beside it link to each section on GitHub.
Eval round — stellar-raven-codemode
This skill is agent-agnostic: it is a plain-markdown runbook. Claude Code invokes it as a skill; Codex or any other CLI agent can be pointed at this file directly.
North star
Every instrument answers one question at some layer:
Does an agent driving this MCP end-to-end produce a correct, current, non-fabricated answer?
Two non-negotiables, from eval/EVALS.md and improvements/README.md (re-read both before
any round — they are the current truth; this skill is the orchestration around them):
- The scores are the instrument; the findings are the product. This server's own tuning
ceiling is single-digit points. The outsized leverage is discovering gaps in the four
upstream surfaces (Lumenloop, Stellar Light/Scout, Stellar Docs, skill sources). A round
that surfaces an upstream gap and doesn't file it in
improvements/has dropped its most valuable output. - One headline, two gates, everything else diagnostic. Never merge lanes, never tune per-question, never promote a view to a gate without a decision recorded in the round ledger.
Agent roles and model boundaries
Do not conflate the agent running this runbook with the model under test:
- Orchestrating agent: Codex/Claude Code/etc. in the repo. It starts servers, runs commands, records result stamps, joins rows with goldens, reviews transcripts, patches code/docs, and files findings. It is not the QA answer model being measured.
- Answering agent: spawned by
eval/qa/run-qa.mjsonce per QA case via headlessclaude -p. It only gets the MCPsearch+executetools and produces the candidate user-facing answer. Default model:claude-sonnet-5, override with--model. - Judge agent: spawned by
eval/qa/judge.mjsto grade the candidate answer against the golden. Default model:claude-sonnet-5, override with--judge-model. Judge verdicts are evidence to review, not unquestionable truth.
There is no committed multi-model matrix unless the round explicitly creates one. Report the answering model, judge model, sample/full-set size, and results-file stamp for every QA run.
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- 3d ago Changed · +47 lines 1ed82e192eeb
- 4d ago Changed · +19 lines 47b079032022
- 7d ago First seen · 659 lines · 137 tokens per session scan A a097f22c92f6
run-evals is a skill published in the GitHub repository stellar-experimental/stellar-raven (6 stars, last pushed 3d ago), licensed Apache-2.0. It adds 137 tokens to every session and 12,101 once invoked, about $0.0007 per session on Opus 5. A static security scan graded it A with 1 finding (makes network calls). No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-31.
Other skills, from other repositories
cli-command
Design, implement, or review Composio CLI commands under ts/packages/cli using Effect, @effect/cli, services, output conventions, configuration and environment variables, and local vendor references. Use for CLI command UX, command wiring, service changes, or CLI source edits. Do not use for CLI E2E-only work; use…
bug-fixing
Fix defects in the Composio SDK repository with focused reproduction, root-cause analysis, regression tests, and narrow verification. Use when the user reports a bug, failing test, CI regression, runtime defect, or incorrect SDK behavior. Do not use for new feature design or broad refactors.
python-sdk
Implement or modify Python SDK behavior under python/composio, including tools, toolkits, sessions, auth configs, connected accounts, client integration, and shared Python models. Use for Python core runtime/API work; pair with python-testing and cross-sdk-parity when TypeScript must match.
python-testing
Select and run Python SDK verification with nox, Makefile targets, Ruff, mypy, pytest markers, sanity tests, type inference checks, and build checks. Use when adding Python tests, diagnosing Python CI, or validating Python SDK/provider changes. Do not use for TypeScript-only checks.
typescript-providers
Implement, modify, test, or document TypeScript provider packages under ts/packages/providers, including framework adapters for OpenAI, Anthropic, Google, LangChain, Mastra, Vercel, LlamaIndex, Cloudflare, and Claude Agent SDK. Use for provider-specific TS work; do not use for core-only changes.
typescript-sdk
Implement or modify TypeScript SDK behavior in @composio/core or shared TypeScript packages, including tools, toolkits, sessions, auth configs, connected accounts, modifiers, and generated SDK surfaces. Use for TS runtime/API work; pair with typescript-testing for verification and cross-sdk-parity when Python must…