benchmark-workflow

benchmark-workflow is a skill for Claude Code, Codex from xberg-io/xberg. It costs 47 tokens per session (522 once invoked), scanned A, original, MIT.

A workflow for running and evaluating Xberg document-extraction benchmarks. Benchmarks compare extraction quality using test documents and independently prepared correct answers.

In plain words
What is it for?
Use it to run benchmark jobs, check ground-truth files, investigate quality changes, manage benchmark fixtures, and work on the benchmark tool.
Why use it?
It helps prevent misleading scores caused by incorrect test answers, mismatched settings, or results that cannot be tied to a specific code version.

Skill for Claude CodeCodex

About the project

Xberg is a document-intelligence engine that reads files, URLs, archives, and source trees and extracts text, metadata, images, tables, and structured data, with additional code-language understanding. Developers use it through language bindings, a command-line tool, REST API, or MCP server, and the catalogue entries support those integrations.

xberg-io/xberg · 9,265 stars · on GitHub

Install

Getting it into your agent

One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.

agentmods
npx agentmods add skills/xberg-io/xberg/benchmark-workflow
Any agent
npx skills add xberg-io/xberg --skill benchmark-workflow
Clone the repo
git clone --depth 1 https://github.com/xberg-io/xberg

Made for: Claude Code, Codex.

Wrote this? Show the measurements

A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.

agentmods badge for benchmark-workflow

README.md
[![agentmods](https://agentmods.dev/badge/skills/xberg-io/xberg/benchmark-workflow.svg)](https://agentmods.dev/skills/xberg-io/xberg/benchmark-workflow)
Your own site
<a href="https://agentmods.dev/skills/xberg-io/xberg/benchmark-workflow"><img src="https://agentmods.dev/badge/skills/xberg-io/xberg/benchmark-workflow.svg" alt="Measured on agentmods" height="20"></a>
Per session 47 Skills are progressive disclosure: only the name and description are preloaded; the body loads when the skill is used.
When invoked 522 The whole file, excluding the scripts and references it only reads on demand.
Security scan A 0 findings. Scan, not verified.
Origin original No closer match found in the catalogue.
Token cost

What it costs to keep this loaded

Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.

ModelPer sessionOnce invoked
Fable 5 $0.00047 $0.00522
Opus 5 $0.00023 $0.00261
Sonnet 5 $0.00009 $0.00104
Haiku 4.5 $0.00005 $0.00052

Measured 5d ago against content hash 6802ba9ffa18, method: parsed. Prices are Anthropic first-party input rates as of 2026-08-30, from the pricing page.

Security

Grade A, and why

benchmark-workflow scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 5d ago.

A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.

Nothing flagged

None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.

.ai-rulez/skills/benchmark-workflow/SKILL.md · 41 lines

How it starts

The opening of the file, as written. The whole thing — 41 lines — stays where its author put it; the contents beside it link to each section on GitHub.

Benchmark workflow

The benchmark system lives in tools/benchmark-harness/; the GitHub workflow is .github/workflows/benchmarks.yaml. The workflow is dispatch-only, so it does not run on push or gate merges. Treat a result as evidence for its exact commit SHA and inputs, not for newer local work.

Ground-truth integrity

  • Never use Xberg's own extractor output as benchmark ground truth. Use an independent source and record it in the fixture's ground_truth.source field (manual, vision, pdf_text_layer, pandoc, python-docx, and similar).
  • Before blaming ground truth for a score, render or otherwise inspect the source document. If the derived .md or .txt disagrees with the source, fix the ground truth; if it agrees, investigate the extractor or metric.
  • Use the fixture schema in tools/benchmark-harness/README.md, the generator at tools/benchmark-harness/scripts/generate_markdown_gt.py, and the harness validate-gt command implemented in tools/benchmark-harness/src/validate_gt.rs. Do not replace these with an ad-hoc conversion pipeline.
  • A quality claim requires the same corpus, config, renderer, cache state, and metric on control and experiment. Disable or invalidate extraction and OCR caches before A/B runs whose output behavior changed.

Diagnosing runs

  • Separate infrastructure failures from extraction or quality failures. A missing backend library, absent fixture, malformed artifact, or runner setup error does not describe extractor quality.
  • Inspect the per-adapter artifacts before the aggregate job. Aggregate contract failures may be caused by missing or unexpectedly named artifacts even when individual adapters ran.
  • Compare accepted OCR pages before raw word counts. Rejected OCR pages contribute neither text nor structured paragraphs.
  • Measure headings and lists using Markdown output. Plain output normalizes away list markers and cannot distinguish detection from rendering.
  • Do not quote a coverage, latency, or quality threshold unless the workflow or harness currently enforces it.

Read the full file on GitHub · 41 lines

Changes

What this file has done since we first saw it

Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.

  1. 5d ago First seen · 41 lines · 47 tokens per session scan A 6802ba9ffa18

Subscribe to this mod's changes

benchmark-workflow is a skill published in the GitHub repository xberg-io/xberg (9,265 stars, last pushed today), licensed MIT. It adds 47 tokens to every session and 522 once invoked, about $0.0002 per session on Opus 5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-30.

Related

Other skills, from other repositories

browser-use

Direct browser control via CDP for web interaction: automation, scraping, testing, screenshots, and site/app work.

browser-use/browser-use · 26 tokens

content-hash-cache-pattern

Cache expensive file processing results using SHA-256 content hashes — path-independent, auto-invalidating, with service layer separation.

affaan-m/ECC · 30 tokens

load-github-action-thread

Download retained Codex GitHub Action thread artifacts and load their rollout history into the local Codex app. Use when asked to open, load, import, resume, or inspect a Codex automation thread from a GitHub Actions run or a related GitHub issue or pull request.

astral-sh/uv · 62 tokens

generating-changelog

Generates polished website release notes between two git tags for docs.streamlit.io. Use when preparing a new Streamlit release or reviewing changes between versions.

streamlit/streamlit · 35 tokens

composio

Route and complete Composio work across Composio For You and Composio Platform. Use when the user mentions Composio; wants an agent to use apps such as Gmail, Slack, GitHub, Notion, Calendar, or Linear; needs first-time setup, an SDK or MCP integration, CLI operation, migration guidance, current documentation, or help…

ComposioHQ/composio · 84 tokens

docs-decisions

Work on Composio documentation content, Fumadocs configuration, changelogs, docs automation prompts, docs decisions, ADR-style records, and docs review guidance. Use for files under docs/, documentation workflows, or requests to record or update a docs decision. Do not use for SDK runtime changes unless docs are the…

ComposioHQ/composio · 72 tokens