skills: Skill for Claude Code

Using the Benchmarking Pipeline

The benchmarking pipeline runs a grid of model/dataset/category combinations end-to-end (train + test) and writes measured metrics to a CSV — use it to produce real, reproducible numbers rather than hand-editing benchmark tables.

Code locations

• src/anomalib/pipelines/benchmark/pipeline.py — Benchmark: top-level pipeline; picks SerialRunner or ParallelRunner based on configured accelerators and torch.cuda.device_count().
• src/anomalib/pipelines/benchmark/generator.py — BenchmarkJobGenerator: expands the config (including grid: entries) into individual jobs.
• src/anomalib/pipelines/benchmark/job.py — BenchmarkJob: runs one model/dataset combination, times it, and saves results.
• tools/experimental/benchmarking/benchmark.py — thin CLI wrapper around Benchmark.
• tools/experimental/benchmarking/sample.yaml — example config to copy from.

Running it

# Via the tools wrapper
python tools/experimental/benchmarking/benchmark.py --config tools/experimental/benchmarking/sample.yaml

# Via the anomalib CLI (registered pipeline subcommand)
anomalib benchmark --config tools/experimental/benchmarking/sample.yaml

Config structure

accelerator:
  - cuda
  - cpu

benchmark:
  seed: 42
  model:
    class_path:
      grid: [Padim, Patchcore]
  data:
    class_path: MVTecAD
    init_args:
      category:
        grid:
          - bottle
          - capsule

Any field can use grid: [...] to sweep multiple values — the generator produces the Cartesian product of every grid field as separate jobs (here: 2 models × 2 categories = 4 jobs). Non-grid fields are held constant across all jobs. data.class_path / model.class_path follow the same anomalib.data.* / anomalib.models.* resolution as everywhere else in the repo (see anomalib-training).

Where results go

BenchmarkJob.save(...) writes one row per job into:

runs/benchmark/<timestamp>/results.csv

(<timestamp> is generated when results are saved via BenchmarkJob.save(), e.g. 2026-08-24-10_30_00.) Each row includes the model/dataset/category combination and the measured metrics — this is the file to consume when building or refreshing README/docs benchmark tables.