Getting it into your agent
One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.
npx skills add Luohaothu/everything-codex --skill evalgit clone --depth 1 https://github.com/Luohaothu/everything-codexWrote this? Show the measurements
A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.
[](https://agentmods.dev/skills/luohaothu/everything-codex/eval)<a href="https://agentmods.dev/skills/luohaothu/everything-codex/eval"><img src="https://agentmods.dev/badge/skills/luohaothu/everything-codex/eval/github.svg" alt="Measured on agentmods" height="20"></a>Or the 80×15 button, for a site that already has a row of RSS and ATOM ones. Only the verdict fits; the numbers stay here.
<a href="https://agentmods.dev/skills/luohaothu/everything-codex/eval"><img src="https://agentmods.dev/badge/skills/luohaothu/everything-codex/eval.svg" alt="Reviewed on agentmods" width="80" height="20"></a>- NVIDIA SkillSpector pass
What it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5.1 | $0.00027 | $0.00818 |
| Opus 5 | $0.00014 | $0.00409 |
| Sonnet 5 | $0.00005 | $0.00164 |
| Haiku 4.5 | $0.00003 | $0.00082 |
Grade A, and why
eval scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 5d ago.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Nothing flagged
None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.
How it starts
The opening of the file, as written. The whole thing — 150 lines — stays where its author put it; the contents beside it link to each section on GitHub.
Eval-Driven Development
A formal evaluation framework implementing eval-driven development (EDD) principles. Evals are the "unit tests of AI development."
Philosophy
- Define expected behavior BEFORE implementation
- Run evals continuously during development
- Track regressions with each change
- Use pass@k metrics for reliability measurement
Usage
/eval define <feature-name> # Create eval definition
/eval check <feature-name> # Run and check evals
/eval report <feature-name> # Generate full report
/eval list # Show all evals
Eval Types
Capability Evals
Test if a new capability works:
[CAPABILITY EVAL: feature-name]
Task: Description of what should be accomplished
Success Criteria:
- [ ] Criterion 1
- [ ] Criterion 2
Expected Output: Description of expected result
Regression Evals
Ensure changes don't break existing functionality:
[REGRESSION EVAL: feature-name]
Baseline: SHA or checkpoint name
Tests:
- existing-test-1: PASS/FAIL
- existing-test-2: PASS/FAIL
Result: X/Y passed
Grader Types
Code-Based Grader (Deterministic)
# Check if file contains expected pattern
grep -q "export function handleAuth" src/auth.ts && echo "PASS" || echo "FAIL"
# Check if tests pass
npm test -- --testPathPattern="auth" && echo "PASS" || echo "FAIL"
Model-Based Grader (Open-ended)
[MODEL GRADER PROMPT]
Evaluate the following code change:
1. Does it solve the stated problem?
2. Is it well-structured?
3. Are edge cases handled?
Score: 1-5
Human Grader
[HUMAN REVIEW REQUIRED]
Change: Description
Risk Level: LOW/MEDIUM/HIGH
Metrics
pass@k — "At least one success in k attempts"
- pass@1: First attempt success rate
- pass@3: Success within 3 attempts
- Target: pass@3 > 90%
pass^k — "All k trials succeed"
- pass^3: 3 consecutive successes
- Use for critical paths
Eval Workflow
1. Define (Before Coding)
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- 5d ago First seen · 150 lines · 27 tokens per session scan A a03a5ed03b6a
eval is a skill published in the GitHub repository Luohaothu/everything-codex (24 stars, last pushed 29d ago), licensed MIT. It adds 27 tokens to every session and 818 once invoked, about $0.0001 per session on Opus 5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-09-03.
Other skills, from other repositories
writing-evals
Scaffolds evaluation suites for the Axiom AI SDK. Generates eval files, scorers, flag schemas, and config from natural-language descriptions. Use when creating evals, writing scorers, setting up flag schemas, or configuring axiom.config.ts.
ai-observability-promptfoo
Testing and evaluation framework for LLM prompts and applications -- promptfooconfig.yaml, assertions, model-graded evals, red teaming, CI/CD integration, custom providers, and comparative evaluation.
evo-simpo-loss
Implement the SimPO (Simple Preference Optimization) loss function for the SimPOTrainer class. Use when reproducing the SimPO paper's loss computation, injecting the simpoloss method into simpotrainer.py, running unit tests, and generating validated loss outputs.
deep-plan
Creates detailed, sectionized, TDD-oriented implementation plans through research, stakeholder interviews, and multi-LLM review. Use when planning features that need thorough pre-implementation analysis.
prompt-optimization-loop
Prompt-engineering expertise for designing test cases, edge cases, adversarial inputs, and iterating on prompts based on eval results.
prompt-evaluation-runner
Use when evaluating prompts, LLM outputs, red-team suites, or model behavior with local eval configs and safe provider/cost controls.