local-test-runner

Local Test Runner

Quick Start

Run vLLM tests locally — either a specific test file or a full Buildkite test area — directly in the current shell with an editable vLLM install.

Buildkite concept	Local equivalent
test_areas/*.yaml steps	pytest commands from those YAML files, run in current env
CI working dir /vllm-workspace/tests	<repo_root>/tests
Buildkite sharding (--num-shards/--shard-id)	Omit locally (run full suite)
$$BUILDKITE_PARALLEL_JOB vars in YAML	Remove or ignore

Workflow

1) Ensure environment is set up

In containerized environments, uv may fail with Cross-device link (os error 18) when its cache and workspace are on different filesystems. If this happens, set the cache to a local directory before any uv commands:

export UV_CACHE_DIR=<repo_root>/.cache/uv

Check that vLLM is fully installed. An import vllm check alone is insufficient: when Python is run from the repo root, the vllm/ source directory shadows the package, so imports can succeed even when uv pip install -e . was never run. In that state there is no vllm console script in .venv/bin/, and entrypoint tests that spawn vllm serve via RemoteOpenAIServer fail with a confusing PermissionError: [Errno 13] Permission denied: 'vllm' (an empty PATH entry causes the exec loop to land on the source directory). All three of these checks must pass:

.venv/bin/python -c "import vllm; print('vllm version:', vllm.__version__)"
test -x .venv/bin/vllm && echo "vllm CLI OK"
uv pip show vllm > /dev/null 2>&1 && echo "pip metadata OK"

If any of the three checks fails, use the install-vllm skill to create the env and install vLLM — do not try to patch around a partial install. Then install test dependencies:

uv pip install -r requirements/test.in
uv pip install -r requirements/dev.txt

1b) Prompt for precompiled wheel before running tests

Always prompt the user for which precompiled wheel to use, unless they have already specified one. The precompiled wheel determines which C++/CUDA binaries are loaded. Using the wrong wheel can cause silent correctness regressions (e.g. a model producing 100% WER instead of ~12%).