infrastructure-benchmark

A deterministic test harness for checking generated project files against written benchmark rules. It reads real files and produces JSON or Markdown score reports.

In plain words
What is it for?
Scoring generated projects against benchmark manifests, refreshing a default template checklist, and checking whether public template outputs meet publication-readiness rules.
Why use it?
It replaces ad-hoc review with repeatable checks whose failures and scores can be inspected. It also keeps readiness checks local, without network or AI calls.

Skill for Claude CodeCodex

Install

Getting it into your agent

One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.

agentmods
npx agentmods add skills/docxology/template/benchmark
Any agent
npx skills add docxology/template --skill benchmark
Clone the repo
git clone --depth 1 https://github.com/docxology/template

Made for: Claude Code, Codex.

Per session 53 Skills are progressive disclosure: only the name and description are preloaded; the body loads when the skill is used.
When invoked 403 The whole file, excluding the scripts and references it only reads on demand.
Security scan A 0 findings. Scan, not verified.
Origin original No closer match found in the catalogue.
Token cost

What it costs to keep this loaded

Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.

ModelPer sessionOnce invoked
Fable 5 $0.00053 $0.00403
Opus 5 $0.00026 $0.00201
Sonnet 5 $0.00011 $0.00081
Haiku 4.5 $0.00005 $0.00040

Measured 2d ago against content hash 5c3c59afd03b, method: parsed. Prices are Anthropic first-party input rates as of 2026-08-30, from the pricing page.

Security

Grade A, and why

infrastructure-benchmark scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 2d ago.

The scan reads SKILL.md. This mod also ships 4 executable files (__init__.py, __main__.py, rubrics.py, …), listed below but not scanned — reading those needs a real analyzer, not pattern matching.

A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.

Nothing flagged

None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.

infrastructure/benchmark/SKILL.md · 49 lines

What it actually says

Benchmark Infrastructure

Use infrastructure.benchmark for small, deterministic readiness benchmarks over public template exemplar outputs. The module reads real files, applies explicit manifest checks and optional weighted rubrics, and emits JSON or Markdown score reports.

Common Workflows

uv run python -m infrastructure.benchmark --repo-root .
uv run python -m infrastructure.benchmark \
  --repo-root . \
  --output-json /tmp/template_benchmark.json \
  --output-markdown /tmp/template_benchmark.md
uv run python -m infrastructure.benchmark \
  --repo-root . \
  --write-default-manifest

Routing Rules

  • Keep benchmark manifests scoped to public template exemplars and generated output contracts.
  • Use real output files, manuscript sources, evidence registries, and artifact manifests.
  • Do not add network calls, LLM calls, mocks, or private-project paths.
  • Keep scoring dimensions explicit and exactly aligned with manifest checks so failed checks, weights, and partial scores remain inspectable.
  • Treat the default manifest as a fail-closed sweep of the full public roster; never shrink it to make a readiness run green.

Public Imports

  • BenchmarkManifest
  • BenchmarkCheckResult
  • BenchmarkScore
  • RubricScore
  • RubricSet
  • load_benchmark_manifest
  • run_benchmark_manifest
  • score_project_against_manifest
  • score_rubric
  • scores_to_dict
  • scores_to_markdown
  • write_default_manifest

Pair this skill with README.md and AGENTS.md for module rules and validation commands.

Files

What ships with it

7 files beside SKILL.md in the same directory: the scripts, references and assets a skill reads on demand. Not counted in the per-session cost; read them before you install if any of them is executable.

Changes

What this file has done since we first saw it

Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.

  1. 2d ago First seen · 49 lines · 53 tokens per session scan A 5c3c59afd03b

Subscribe to this mod's changes

infrastructure-benchmark is a skill published in the GitHub repository docxology/template (19 stars, last pushed 2d ago), licensed Apache-2.0. It adds 53 tokens to every session and 403 once invoked, about $0.0003 per session on Opus 5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-30.

Related

Other skills, from other repositories

browser-trace

Capture a full DevTools-protocol trace of any browser automation — CDP firehose, screenshots, and DOM dumps — then bisect the stream into per-page searchable buckets. Use when the user wants to debug a failed run, audit network/console/DOM activity, attach a trace to an in-progress session, or feed structured per-page…

mxyhi/ok-skills · 88 tokens

planning-with-files

Manus-style persistent file-based planning for AI coding agents: keeps taskplan.md, findings.md, and progress.md on disk so work survives context loss and /clear. Use when asked to plan out, break down, or organize a multi-step project, research task, or any work requiring 5+ tool calls. Supports automatic session…

mxyhi/ok-skills · 76 tokens

ai-elements

Build AI chat interfaces using ai-elements components — conversations, messages, tool displays, prompt inputs, and more. Use when the user wants to build a chatbot, AI assistant UI, or any AI-powered chat interface.

mxyhi/ok-skills · 46 tokens

exa-search

Use Exa MCP for current web, code/docs, company, people, and page-fetch research. Prefer current hosted tool schemas and note deprecated tools.

mxyhi/ok-skills · 33 tokens

ontoly-software-graph

Use Ontoly's deterministic Software Graph and MCP capabilities for repository architecture, request tracing, dependency analysis, configuration lookup, and impact analysis before falling back to source search.

mxyhi/ok-skills · 40 tokens

kimi-webbridge

Kimi WebBridge lets AI control the user's real browser — navigate, click, type, read, screenshot, and interact with any website using the user's actual login sessions. Use this skill whenever the user wants to interact with websites, automate browser tasks, scrape web content, or perform any action requiring a real…

mxyhi/ok-skills · 113 tokens