clawbench-base

ClawBench

ClawBench is the OpenClaw evaluation workflow exposed through ClawMind and ClawWeb. It runs evaluation domains, records task results, captures transcripts, and can generate diagnostic reports.

Do not expose internal benchmark runtime names to end users. Refer to this capability as ClawBench in user-facing responses, UI copy, docs, and release material.

When to Use

• The user wants to run an OpenClaw evaluation with /clawbench.
• The user wants to repair or regenerate a ClawBench report with /clawbenchreport.
• The user asks whether a ClawBench run was intercepted by ClawMind.
• The user wants to locate run status, result files, transcripts, or diagnosis output.
• The user reports that ClawBench is pending, stuck, or failed.
• The user needs a publishing-safe explanation of the ClawBench workflow.

Common Commands

Run all templates in a domain:

/clawbench run --domainId <domain_id>

Run one template:

/clawbench run --domainId <domain_id> --templateName <template_name>

Run as an explicit ClawWeb owner on a shared evaluation machine:

/clawbench run --domainId <domain_id> --ownerId <user_id>

Repair or regenerate the report for an existing run:

/clawbenchreport run --benchId <bench_run_id>

Useful local checks:

tail -n 200 ~/.openclaw/logs/gateway.log
curl --noproxy '*' -s http://localhost:3001/api/bench/runs/<benchRunId>
find ~/.openclaw/workspace -path '*benchmark*' -maxdepth 8 -type f

Included Runtime Files

This skill ships the local ClawBench runner implementation:

• scripts/: runner, adapter, grading, upload, and utility scripts.
• tests/: core library tests.
• report/: report template and report-generation skill source.

Task datasets are intentionally not packaged in this skill. Runtime tasks are expected to come from ClawWeb/centralized configuration.

Execution Path

1. OpenClaw receives /clawbench ....
2. ClawMind resolves the ClawBench facade binding. Report repair uses the separate /clawbenchreport facade.
3. ClawMind loads the workflow spec from ClawWeb API/DB.
4. Normal workflow creates a ClawWeb bench run record. Report repair workflow resolves an existing bench run instead.
5. Workflow runs the local evaluation engine.
6. Workflow uploads result artifacts and task summaries to ClawWeb.
7. Workflow prepares report context and optionally runs clawbench-report.
8. ClawWeb displays run status, score, pass rate, task records, and report summary.