vllm-xpu-profile

vllm-xpu-profile

Profile a vLLM-XPU server with torch.profiler to see scheduler, KV manager, attention backend, and batching alongside XPU op timeline. For pure-PyTorch traces use torch-xpu-profile; for SYCL kernel level use xpu-profile-unitrace.

Modes

Mode A — server + HTTP bracket

Launch the server with profiler flags. Always launch from the official upstream vllm/vllm-openai-xpu:latest image — even if a running container or host process is using a different image (e.g. intel/llm-scaler-vllm), do not reuse that image for the profiling server; those stacks are out of scope (see What this skill does NOT cover).

docker run -d --name vllm-xpu-prof \
    --device /dev/dri \
    -v /dev/dri/by-path:/dev/dri/by-path:ro \
    --group-add "$(getent group render | cut -d: -f3)" \
    --ipc=host \
    -e ZE_AFFINITY_MASK=0 \
    -e VLLM_WORKER_MULTIPROC_METHOD=spawn \
   -e HTTP_PROXY -e HTTPS_PROXY -e NO_PROXY \
   -e http_proxy -e https_proxy -e no_proxy \
   -e HF_TOKEN \
    -v "$HOME/.cache/huggingface:/root/.cache/huggingface" \
    -v "$PWD/traces:/work/traces" \
    -p 8000:8000 \
   vllm/vllm-openai-xpu:latest \
   Qwen/Qwen2.5-1.5B-Instruct \
        --dtype bfloat16 --enforce-eager --max-model-len 4096 \
        --profiler-config.profiler=torch \
        --profiler-config.torch_profiler_dir=/work/traces

Wait for Application startup complete, then bracket your window:

# Warmup (don't profile this) — kernel cache hot
for i in 1 2 3; do
    curl -s http://localhost:8000/v1/chat/completions \
        -H 'Content-Type: application/json' \
        -d '{"model":"Qwen/Qwen2.5-1.5B-Instruct",
             "messages":[{"role":"user","content":"hi"}],"max_tokens":32}' >/dev/null
done

curl -X POST http://localhost:8000/start_profile

# Workload to characterise
for i in $(seq 1 8); do
    curl -s http://localhost:8000/v1/chat/completions \
        -H 'Content-Type: application/json' \
        -d '{"model":"Qwen/Qwen2.5-1.5B-Instruct",
             "messages":[{"role":"user","content":"Write a short paragraph."}],
             "max_tokens":128}' &
done; wait

curl -X POST http://localhost:8000/stop_profile
ls traces/