AAS Core is a local control plane for coding agents that lets them search a large catalogue of skills, choose a stack, validate it, and create a reproducible plan. It is used to assemble and review agent workflows through its CLI, local MCP server, catalogue, plugins, and Workbench. The catalogue add-ons provide the skills, plugins, bundles, and workflows that AAS Core helps agents select and validate.
Getting it into your agent
One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.
npx skills add sickn33/agentic-awesome-skills --skill ai-agent-evaluation-benchmarkinggit clone --depth 1 https://github.com/sickn33/agentic-awesome-skillsWrote this? Show the measurements
A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.
[](https://agentmods.dev/skills/sickn33/agentic-awesome-skills/ai-agent-evaluation-benchmarking)<a href="https://agentmods.dev/skills/sickn33/agentic-awesome-skills/ai-agent-evaluation-benchmarking"><img src="https://agentmods.dev/badge/skills/sickn33/agentic-awesome-skills/ai-agent-evaluation-benchmarking/github.svg" alt="Measured on agentmods" height="20"></a>Or the 80×15 button, for a site that already has a row of RSS and ATOM ones. Only the verdict fits; the numbers stay here.
<a href="https://agentmods.dev/skills/sickn33/agentic-awesome-skills/ai-agent-evaluation-benchmarking"><img src="https://agentmods.dev/badge/skills/sickn33/agentic-awesome-skills/ai-agent-evaluation-benchmarking.svg" alt="Reviewed on agentmods" width="80" height="20"></a>What it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5.1 | $0.00031 | $0.01375 |
| Opus 5.5 | $0.00012 | $0.00550 |
| Sonnet 5.5 | $0.00006 | $0.00275 |
| Haiku 4.5 | $0.00003 | $0.00137 |
Grade A, and why
ai-agent-evaluation-benchmarking scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured yesterday.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Nothing flagged
None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.
Copies of this mod
2 near-identical copies found in the catalogue:
- ai-agent-evaluation-benchmarking — 100% identical, 25 lines differ
- ai-agent-evaluation-benchmarking — 100% identical, 0 lines differ
How it starts
The opening of the file, as written. The whole thing — 130 lines — stays where its author put it; the contents beside it link to each section on GitHub.
AI Agent Capability Evaluation & Benchmarking
What it is: Standardizes multi-metric capability benchmarking, token cost efficiency, and regression monitoring across autonomous coding agents.
Overview
Provides a standardized, auditable framework and data model for AI Agent Capability Evaluation & Benchmarking operations across distributed engineering and decentralized application systems.
When to Use This Skill
- When recording agent evaluation runs with a fixed dataset, rubric, runtime/model revision and measured outcomes.
- When comparing a candidate with a baseline under matched tasks, environment, tool permissions and cost accounting.
- When generating reproducible CSV, SQL DDL, JSON Schema, and Notion property registers for tracking compliance.
How It Works
- Define the parameters, thresholds, and identity bindings required for the target operational register.
- Select appropriate boundary enforcement values from validated enum select sets.
- Export standardized artifacts (CSV table, SQL DDL, JSON Schema) to integrate into validation CI pipelines.
Field Reference
| # | Field Name | Type | SQL Type | JSON Schema Type | Notion Property Type | Example Value |
|---|---|---|---|---|---|---|
| 1 | Benchmark Run ID | id |
SERIAL PRIMARY KEY |
integer |
Text | 1 |
| 2 | Evaluated Agent Model | select |
VARCHAR(64) |
string |
Select | Claude 3.7 Sonnet |
| 3 | Benchmark Suite Domain | select |
VARCHAR(64) |
string |
Select | SWE-bench Verified |
| 4 | Tasks Evaluated Count | number |
INTEGER |
integer |
Number | 100 |
| 5 | Pass Rate Percentage | number |
NUMERIC(5,2) |
number |
Number | 78.40 |
| 6 | Tool Hallucination Rate % | number |
NUMERIC(5,2) |
number |
Number | 0.60 |
| 7 | Average Tokens Per Task | number |
INTEGER |
integer |
Number | 42500 |
| 8 | Cost Per Solved Task USD | currency |
NUMERIC(8,4) |
number |
Number | 0.3420 |
| 9 | Regression Verdict | select |
VARCHAR(32) |
string |
Select | Superior |
| 10 | Evaluation Lead | text |
VARCHAR(64) |
string |
Text | Ranjeet2063 |
| 11 | Benchmark Execution Date | date |
DATE |
string, format: date |
Date | 2026-10-01 |
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- yesterday Changed · +5 lines 952617fd1a23
- 13d ago First seen · 125 lines · 31 tokens per session scan A 39f227373167
ai-agent-evaluation-benchmarking is a skill published in the GitHub repository sickn33/agentic-awesome-skills (47,686 stars, last pushed today), licensed MIT. It adds 31 tokens to every session and 1,375 once invoked, about $0.0001 per session on Opus 5.5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-10-03.
Other skills, from other repositories
ai-agent-evaluation-benchmarking
Autonomous AI agent benchmark evaluation register: task completion rates, planning accuracy, tool invocation precision, and cost benchmarks.
ai-agent-evaluation-benchmarking
Autonomous AI agent benchmark evaluation register: task completion rates, planning accuracy, tool invocation precision, and cost benchmarks.
ai-engineering-toolkit
6 production-ready AI engineering workflows: prompt evaluation (8-dimension scoring), context budget planning, RAG pipeline design, agent security audit (65-point checklist), eval harness building, and product sense coaching.
trulens-diagnosis
Diagnose low evaluation scores and generate actionable improvement recommendations.
trulens-running-evaluations
Execute TruLens evaluations and view results.
cocotrace
Build and query the SHA-256 requirements-to-implementation traceability graph across lifecycle artifacts.