run-llm-d-benchmark

Run llm-d Benchmark

Purpose

Run a benchmark workload against an already-deployed llm-d stack using the llmdbenchmark CLI. Useful for evaluating the performance of the llm-d stack. Supports two modes:

• Guide mode: the stack was deployed from a named llm-d guide (e.g. optimized-baseline). Use --spec guides/<name> and pick from guide-specific or generic workload profiles.
• Custom workload mode: the user provides their own workload profile (or a path to an existing one) and a direct endpoint URL. No guide name required.

Both modes follow helpers/benchmark.md. When the benchmark completes, results are available in the local workspace directory.

---

Workflow

Step 1: Verify or Install the llmdbenchmark CLI

Check whether the CLI is available:

command -v llmdbenchmark 2>/dev/null && llmdbenchmark --version

If not found, check for the repo locally and install if missing:

# Is the repo already cloned?
ls ./llm-d-benchmark 2>/dev/null || \
  curl -sSL https://raw.githubusercontent.com/llm-d/llm-d-benchmark/main/install.sh | bash

For troubleshooting install issues - Check Makefile for install instructions and dependencies.

Activate the virtualenv and enter the repo — both are required for every new shell session:

cd llm-d-benchmark
source .venv/bin/activate
llmdbenchmark --version

All subsequent llmdbenchmark commands must be run from inside llm-d-benchmark/ with the venv active.

---

Step 2: Locate the Namespace and (Optionally) the Guide Name

Locate the llm-d stack according to the following logic:

1. If a NAMESPACE environment variable is specified, the llm-d stack is assumed to be deployed there
2. If an oc project exists, the stack is assumed to be deployed in the current oc project.
3. If none of the above holds, ask the user for the NAMESPACE where it is deployed.
4. Make sure the NAMESPACE environment variable is set.