benchmarks: Skill for Claude Code

Local end-to-end: @benchsdk/runner ↔ benchmarks-platform

Goal: exercise upsert benchmark → create run → planWorkers → claim → heartbeat → task_results → artifact upload → complete → dashboard, with zero external credentials.

1. Infra (docker)

docker run -d --name pg -p 5432:5432 -e POSTGRES_PASSWORD=postgres -e POSTGRES_DB=bench postgres:16
docker run -d --name minio -p 9000:9000 -e MINIO_ROOT_USER=minioadmin -e MINIO_ROOT_PASSWORD=minioadmin \
  quay.io/minio/minio server /data
docker run -d --name ch -p 8123:8123 -e CLICKHOUSE_PASSWORD=chpass clickhouse/clickhouse-server:25.6
docker run --rm --network host --entrypoint sh quay.io/minio/mc -c \
  "mc alias set local http://127.0.0.1:9000 minioadmin minioadmin && mc mb -p local/bench"

Substitutes when images or ports aren't available:

• Postgres port taken (a host postgres already on 5432): use it — DATABASE_URL=postgresql://ci:[email protected]:5432/ci (the CI creds; the ci database works when CREATE DATABASE is denied).

• MinIO image won't pull (pull access denied / quay 401): s3rver is a drop-in — npx -y s3rver -p 9000 -a 0.0.0.0 --directory ~/s3data, then AWS_ACCESS_KEY_ID=S3RVER AWS_SECRET_ACCESS_KEY=S3RVER aws --endpoint-url http://127.0.0.1:9000 s3 mb s3://bench and set TIGRIS_ACCESS_KEY_ID/ TIGRIS_SECRET_ACCESS_KEY=S3RVER in .env.local.

• npm run db:migrate says DATABASE_URL ... is not set: it doesn't read .env.local — run with set -a; . ./.env.local; set +a; npm run db:migrate.

• Minting /mcp access tokens for MCP E2E: insert a jwks row (EdDSA keypair, both JWKs stored), an oauth_client row, and an oauth_consent row (clientId + userId + referenceId=orgId, non-null scopes), then sign an at+jwt (kid = the jwks row id) with claims sub=user, client_id, org, scope, iss=$BETTER_AUTH_URL/api/auth, aud=$BETTER_AUTH_URL/mcp — see lib/oauth/tokens.ts verifyAccessToken for the exact shape.

• Use ClickHouse >= 25.x: 24.8 fails ch:migrate with TTL expression result column should have DateTime or Date type, but has DateTime64(3,'UTC').

• The importer passes clickhouse_settings: { date_time_input_format: 'best_effort' } per insert, so a default-configured server works. If you hit Cannot parse input: expected '"' before: 'Z"...' (older code), work around it server-side, and remember to REMOVE the override before verifying an importer fix — otherwise the server setting masks it:

  docker exec ch bash -c 'mkdir -p /etc/clickhouse-server/users.d && printf "<clickhouse><profiles><default><date_time_input_format>best_effort</date_time_input_format></default></profiles></clickhouse>" > /etc/clickhouse-server/users.d/besteffort.xml'
  docker restart ch
  # verify which mode is actually active:
  curl -s "http://127.0.0.1:8123/?user=default&password=chpass" \
    --data-binary "SELECT value FROM system.settings WHERE name='date_time_input_format'"