benchmark-workflow

benchmark-workflow is a skill for Claude Code, Codex from xberg-io/xberg. It costs 43 tokens per session (547 once invoked), scanned A, original, MIT.

A workflow for running and evaluating Xberg document-extraction benchmarks. Benchmarks compare extraction quality using test documents and independently prepared correct answers.

In plain words
What is it for?
Use it to run benchmark jobs, check ground-truth files, investigate quality changes, manage benchmark fixtures, and work on the benchmark tool.
Why use it?
It helps prevent misleading scores caused by incorrect test answers, mismatched settings, or results that cannot be tied to a specific code version.

Skill for Claude CodeCodex

Written for no agent in particular: nothing here depends on one.

Good fit Use it to run benchmark jobs, check ground-truth files, investigate quality changes, manage benchmark fixtures, and work on the benchmark tool.

Compare 6 skills from other repositories ↓
Install with agentmods
npx agentmods add skills/xberg-io/xberg/benchmark-workflow
About the project

Xberg is a document-intelligence engine that reads files, URLs, archives, and source trees and extracts text, metadata, images, tables, and structured data, with additional code-language understanding. Developers use it through language bindings, a command-line tool, REST API, or MCP server, and the catalogue entries support those integrations.

xberg-io/xberg · 9,428 stars · on GitHub · docs.xberg.io

Install

Getting it into your agent

One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.

Any agent
npx skills add xberg-io/xberg --skill benchmark-workflow
Clone the repo
git clone --depth 1 https://github.com/xberg-io/xberg

Made for: Claude Code, Codex.

Wrote this? Show the measurements

A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.

agentmods badge for benchmark-workflow

README.md
[![agentmods](https://agentmods.dev/badge/skills/xberg-io/xberg/benchmark-workflow/github.svg)](https://agentmods.dev/skills/xberg-io/xberg/benchmark-workflow)
Your own site
<a href="https://agentmods.dev/skills/xberg-io/xberg/benchmark-workflow"><img src="https://agentmods.dev/badge/skills/xberg-io/xberg/benchmark-workflow/github.svg" alt="Measured on agentmods" height="20"></a>

Or the 80×15 button, for a site that already has a row of RSS and ATOM ones. Only the verdict fits; the numbers stay here.

agentmods 80×15 button for benchmark-workflow

Your own site · 80×15
<a href="https://agentmods.dev/skills/xberg-io/xberg/benchmark-workflow"><img src="https://agentmods.dev/badge/skills/xberg-io/xberg/benchmark-workflow.svg" alt="Reviewed on agentmods" width="80" height="20"></a>
Per session 43 Skills are progressive disclosure: only the name and description are preloaded; the body loads when the skill is used.
When invoked 547 The whole file, excluding the scripts and references it only reads on demand.
Security scan A 0 findings. A grade says what 26 rules found in the file — not that it is safe. Third-party audits
  • NVIDIA SkillSpector pass 7 Sept 2026
How audits are shown
Origin original No closer match found in the catalogue.
Token cost

What it costs to keep this loaded

Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.

ModelPer sessionOnce invoked
Fable 5.1 $0.00043 $0.00547
Opus 5.5 $0.00017 $0.00219
Sonnet 5.5 $0.00009 $0.00109
Haiku 4.5 $0.00004 $0.00055

Measured 8d ago against content hash ce8e27ce2d8e, method: parsed. Prices are Anthropic first-party input rates as of 2026-10-07, from the pricing page.

Security

Grade A, and why

benchmark-workflow scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 8d ago.

A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.

Nothing flagged

None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.

.ai-rulez/skills/benchmark-workflow/SKILL.md · 44 lines

How it starts

The opening of the file, as written. The whole thing — 44 lines — stays where its author put it; the contents beside it link to each section on GitHub.

Benchmark workflow

The benchmark system lives in tools/benchmark-harness/; the GitHub workflow is .github/workflows/benchmarks.yaml. The workflow is dispatch-only, so it does not run on push or gate merges. Treat a result as evidence for its exact commit SHA and inputs, not for newer local work.

Ground-truth integrity

  • Never use Xberg's own extractor output as benchmark ground truth. Use an independent source and record it in the fixture's ground_truth.source field (manual, vision, pdf_text_layer, pandoc, python-docx, and similar).
  • Before blaming ground truth for a score, render or otherwise inspect the source document. If the derived .md or .txt disagrees with the source, fix the ground truth; if it agrees, investigate the extractor or metric.
  • Use the fixture schema in tools/benchmark-harness/README.md, the generator at tools/benchmark-harness/scripts/generate_markdown_gt.py, and the harness validate-gt command implemented in tools/benchmark-harness/src/validate_gt.rs. Do not replace these with an ad-hoc conversion pipeline.
  • A quality claim requires the same corpus, config, renderer, cache state, and metric on control and experiment. Disable or invalidate extraction and OCR caches before A/B runs whose output behavior changed.

Diagnosing runs

  • Separate infrastructure failures from extraction or quality failures. A missing backend library, absent fixture, malformed artifact, or runner setup error does not describe extractor quality.
  • Inspect the per-adapter artifacts before the aggregate job. Aggregate contract failures may be caused by missing or unexpectedly named artifacts even when individual adapters ran.
  • Compare accepted OCR pages before raw word counts. Rejected OCR pages contribute neither text nor structured paragraphs.
  • Measure headings and lists using Markdown output. Plain output normalizes away list markers and cannot distinguish detection from rendering.
  • Do not quote a coverage, latency, or quality threshold unless the workflow or harness currently enforces it.

Read the full file on GitHub · 44 lines

Files

What ships with it

1 file beside SKILL.md in the same directory: the scripts, references and assets a skill reads on demand. Not counted in the per-session cost; read them before you install if any of them is executable.

Changes

What this file has done since we first saw it

Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.

  1. 8d ago Changed · +3 lines · -4 tokens per session ce8e27ce2d8e
  2. 1mo ago First seen · 41 lines · 47 tokens per session scan A 6802ba9ffa18

Subscribe to this mod's changes

benchmark-workflow is a skill published in the GitHub repository xberg-io/xberg (9,428 stars, last pushed yesterday), licensed MIT. It adds 43 tokens to every session and 547 once invoked, about $0.0002 per session on Opus 5.5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-30.

Related

Other skills, from other repositories

browser-use

Direct browser control via CDP for web interaction: automation, scraping, testing, screenshots, and site/app work. Also covers hosted cloud browsers for AI agents: the browser-use Python library (Browser(usecloud=True)), Playwright, Puppeteer or any CDP client.

browser-use/browser-use · 57 tokens

credit-note-fixer

Fix the tiny credit-note formatting bug and rerun the exact targeted test command.

openai/openai-agents-python · 21 tokens

typescript-testing

Select and run TypeScript SDK verification for packages, examples, type checks, linting, builds, Vitest suites, Effect v4 CLI tests, and runtime E2E tests. Use when adding tests, diagnosing TypeScript CI, choosing a focused test command, validating TypeScript package changes, or writing/porting CLI tests against…

ComposioHQ/composio · 91 tokens

bug-fixing

Fix defects in the Composio SDK repository with focused reproduction, root-cause analysis, regression tests, and narrow verification. Use when the user reports a bug, failing test, CI regression, runtime defect, or incorrect SDK behavior. Do not use for new feature design or broad refactors.

ComposioHQ/composio · 63 tokens

python-testing

Select and run Python SDK verification with nox, Makefile targets, Ruff, mypy, pytest markers, sanity tests, type inference checks, and build checks. Use when adding Python tests, diagnosing Python CI, or validating Python SDK/provider changes. Do not use for TypeScript-only checks.

ComposioHQ/composio · 62 tokens

cli-e2e

Write, modify, or debug Docker-based Composio CLI end-to-end tests under ts/e2e-tests/cli, including binary invocation, fixture isolation, output assertions, and package manifests. Use for CLI E2E test suites only; use cli-command for CLI source implementation.

ComposioHQ/composio · 62 tokens