vLLM Semantic Router is a programmable routing layer that chooses or combines language models for each request in a system using multiple models and types of computing infrastructure. It helps teams route inference by signals such as user preferences, application policies, quality, cost, latency, privacy, and safety requirements. The catalogue skills and instructions support configuring and operating this model-routing system.
Getting it into your agent
One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.
npx skills add vllm-project/semantic-router --skill vllm-srgit clone --depth 1 https://github.com/vllm-project/semantic-routerWrote this? Show the measurements
A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.
[](https://agentmods.dev/skills/vllm-project/semantic-router/vllm-sr)<a href="https://agentmods.dev/skills/vllm-project/semantic-router/vllm-sr"><img src="https://agentmods.dev/badge/skills/vllm-project/semantic-router/vllm-sr/github.svg" alt="Measured on agentmods" height="20"></a>Or the 80×15 button, for a site that already has a row of RSS and ATOM ones. Only the verdict fits; the numbers stay here.
<a href="https://agentmods.dev/skills/vllm-project/semantic-router/vllm-sr"><img src="https://agentmods.dev/badge/skills/vllm-project/semantic-router/vllm-sr.svg" alt="Reviewed on agentmods" width="80" height="20"></a>What it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5.1 | $0.00071 | $0.04473 |
| Opus 5.5 | $0.00028 | $0.01789 |
| Sonnet 5.5 | $0.00014 | $0.00895 |
| Haiku 4.5 | $0.00007 | $0.00447 |
Grade C, and why
vllm-sr scanned grade C with 3 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 6d ago.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Sends data to an external URLmediumData exfiltration
A POST to an outside endpoint may be telemetry or may be exfiltration; either way the mod talks to somewhere, and you should know where.
curl -fsS http://127.0.0.1:9099/v1/systemone -H 'content-type: application/json' -d '{ Asks for rootmediumPrivilege escalation
A mod that escalates privileges can change anything on the machine, not only the project.
| Python | 3.10 or newer, and `ensurepip` prints `pip …` | Ubuntu and Debian: with the user's OK, `sudo apt-get install -y python3-venv`. Without it the installer stops at "ensurepip is not available". | Makes network callslowCapability
Not a fault in itself. Listed so you know the mod talks to something, and to what.
curl -s -m 3 http://127.0.0.1:11434/api/tags # Ollama: its model names How it starts
The opening of the file, as written. The whole thing — 343 lines — stays where its author put it; the contents beside it link to each section on GitHub.
vLLM Semantic Router: install, configure, verify
Run the steps in order. Each one says what to run, what success looks like and what to do otherwise, and each is safe to run again.
The Router serves an OpenAI-compatible API and picks a model for each request.
It does not run the models that answer users; those are the user's endpoints,
such as vLLM, Ollama or a hosted API. vllm-sr serve runs a stack in Docker:
the Router (inference on port 8899, management API on 8080), the Dashboard
(8700), the model runtime for the Router's own classifiers, and supporting
services.
Ground rules
- Keep secrets in environment variables, never in YAML, command arguments, logs or your report.
- Don't stop, restart or replace what you didn't start (containers, a running
vllm-srstack, services on its ports) without asking. - Ask before system-wide changes: installing packages, adding the user to a group, publishing a port beyond loopback.
- Never run
vllm-sr servewithout a complete--config. Without one it starts the Dashboard's setup mode and waits for a person indefinitely. - An HTTP 200 alone is not success. Each check below names what the output must contain.
1. Preflight
These commands change nothing. Run them and keep the results.
uname -sm
docker info --format '{{.ServerVersion}}'
python3 -c 'import sys; print(sys.version.split()[0])'
python3 -m ensurepip --version
df -h "$HOME" | tail -1
ls /dev/kfd /dev/dri/render* 2>/dev/null
nvidia-smi -L 2>/dev/null
vllm-sr --version 2>/dev/null && vllm-sr status
| Check | Pass | Otherwise |
|---|---|---|
| Docker | prints a version | "permission denied": ask the user to add their account to the docker group. Not installed: ask before installing it. Podman works too: add --container-runtime podman to vllm-sr commands. |
| Python | 3.10 or newer, and ensurepip prints pip … |
Ubuntu and Debian: with the user's OK, sudo apt-get install -y python3-venv. Without it the installer stops at "ensurepip is not available". |
| Disk | 10 GB free; 30 GB for a GPU platform | Ask the user. |
vllm-sr |
not installed, or status says Not running |
A stack is already running. It is the user's: find its directory (below), verify it with step 6 using its own decision and model names, and ask before changing it. |
What ships with it
6 files beside SKILL.md in the same directory: the scripts, references and assets a skill reads on demand. Not counted in the per-session cost; read them before you install if any of them is executable.
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- 6d ago Changed a7b7d282c8bf
- 20d ago Changed · +9 lines ac3c86f4c5ff
- 22d ago Changed · +278 lines · +24 tokens per session scan A → C 9dda5c50ee51
- 1mo ago Changed c090192c2671
- 1mo ago Changed · -76 lines · +8 tokens per session fd87a0477b56
- 1mo ago Changed · -119 lines · -7 tokens per session ecb8aa30ae6b
- 1mo ago Changed · +8 lines d1c6545679a0
- 1mo ago Changed · +106 lines 6f64cec81415
- 1mo ago Changed · +28 lines 7728d15f6601
- 1mo ago First seen · 109 lines · 46 tokens per session scan A 3d7f1cd79ff4
vllm-sr is a skill published in the GitHub repository vllm-project/semantic-router (6,328 stars, last pushed today), licensed Apache-2.0. It adds 71 tokens to every session and 4,473 once invoked, about $0.0003 per session on Opus 5.5. A static security scan graded it C with 3 findings (sends data to an external url, asks for root, makes network calls). No closer match exists in the catalogue, so it is treated as the original; first seen 2026-09-12.
Other skills, from other repositories
runpod
Use when running GPU compute on RunPod and deciding between Pods (hourly, always-on) and Serverless (per-second, autoscaling) for training, fine-tuning or inference — serverless endpoints, handler workers, worker Docker templates, network volumes, timeout and worker-count tuning, cold starts, and runaway bills. NOT…
serving-llms-vllm
Serves LLMs with high throughput using vLLM's PagedAttention and continuous batching. Use when deploying production LLM APIs, optimizing inference latency/throughput, or serving models with limited GPU memory. Supports OpenAI-compatible endpoints, quantization (GPTQ/AWQ/FP8), and tensor parallelism.
ecs-genai
Use whenever someone runs a GPU / ML / GenAI / LLM workload on Amazon ECS: GPU on ECS, ECS GPU-optimized AMI, g4dn/g5/g6/p4/p5 on ECS, which ECS launch type for GPU, Inferentia/Trainium/Neuron on ECS, distributed training, model inference or vLLM/Triton/TGI/Ray on ECS, Capacity Blocks, GPU sharing, ASG per GPU type.…
eks-genai
Use whenever someone is building, training, fine-tuning, or serving a generative AI / LLM workload on Amazon EKS — phrased as "GPU vs Trainium/Inferentia", "vLLM on EKS", "Ray Serve / KubeRay", "distributed training on EKS", "FSx for Lustre for ML", "Karpenter for GPU", "EFA / NCCL multi-node", "DCGM / Neuron…
litellm-setup
Set up a self-hosted LiteLLM proxy, the OpenAI-compatible LLM gateway, on this machine, from install through provider keys, config.yaml, a master key, optional Postgres and Admin UI, and a verified chat completion. Use when the user asks to set up LiteLLM, install the LiteLLM proxy, self-host an LLM gateway, or put…
tensorrt-llm
High-throughput LLM inference on NVIDIA GPUs.