vLLM Semantic Router is a programmable routing layer that chooses or combines language models for each request in a system using multiple models and types of computing infrastructure. It helps teams route inference by signals such as user preferences, application policies, quality, cost, latency, privacy, and safety requirements. The catalogue skills and instructions support configuring and operating this model-routing system.
Getting it into your agent
One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.
npx skills add vllm-project/semantic-router --skill vllm-srgit clone --depth 1 https://github.com/vllm-project/semantic-routerWrote this? Show the measurements
A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.
[](https://agentmods.dev/skills/vllm-project/semantic-router/vllm-sr)<a href="https://agentmods.dev/skills/vllm-project/semantic-router/vllm-sr"><img src="https://agentmods.dev/badge/skills/vllm-project/semantic-router/vllm-sr/github.svg" alt="Measured on agentmods" height="20"></a>Or the 80×15 button, for a site that already has a row of RSS and ATOM ones. Only the verdict fits; the numbers stay here.
<a href="https://agentmods.dev/skills/vllm-project/semantic-router/vllm-sr"><img src="https://agentmods.dev/badge/skills/vllm-project/semantic-router/vllm-sr.svg" alt="Reviewed on agentmods" width="80" height="20"></a>What it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5.1 | $0.00046 | $0.01007 |
| Opus 5 | $0.00023 | $0.00504 |
| Sonnet 5 | $0.00009 | $0.00201 |
| Haiku 4.5 | $0.00005 | $0.00101 |
Grade A, and why
vllm-sr scanned grade A with 1 finding against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured today.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Makes network callslowCapability
Not a fault in itself. Listed so you know the mod talks to something, and to what.
curl -fsSL https://vllm-sr.ai/install.sh | \ How it starts
The opening of the file, as written. The whole thing — 109 lines — stays where its author put it; the contents beside it link to each section on GitHub.
vLLM Semantic Router
Use the CLI and Router API directly. The Dashboard is optional and must not be a dependency of this workflow. The user's instructions and deployment boundaries take precedence over this skill.
Authoritative contracts
- Treat the installed CLI help, the running Router's discovery response, JSON Schema, and OpenAPI document as the current source of truth.
- Discover progressively. Start with
vllm-sr config schema, then request only the relevant--sectionor--surface. Use--fullonly when the complete contract is required. - Discover Router operations from
GET /api/v1; fetch the full or filtered OpenAPI document from/openapi.jsonwhen request and response details are needed. - Do not reuse remembered fields or endpoints when runtime discovery is available.
Workflow
-
Clarify the requested outcome and inspect the host, existing installation, current config, model endpoints, container runtime, and accelerator with read-only commands.
-
If the CLI is missing, install the stable release without starting or changing a runtime yet:
curl -fsSL https://vllm-sr.ai/install.sh | \ bash -s -- --channel stable --mode cli --runtime skip --no-launch vllm-sr --version -
Inspect
vllm-sr serve --helpand select the deployment path that matches the actual host. Do not assume a GPU platform. Do not expose a management listener publicly unless the user explicitly requests and secures it. -
Discover the config surface before writing YAML:
vllm-sr config schema vllm-sr config schema --section providers vllm-sr config schema --section routingUse
vllm-sr config schema --surface KIND:NAMEfor a selected signal, projection, algorithm, or plugin. -
Create or update canonical YAML. Keep credentials in environment variables and store only environment references in the config.
-
Validate locally, then ask the running Router to plan the exact mutation:
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- today First seen · 109 lines · 46 tokens per session scan A 3d7f1cd79ff4
vllm-sr is a skill published in the GitHub repository vllm-project/semantic-router (5,741 stars, last pushed today), licensed Apache-2.0. It adds 46 tokens to every session and 1,007 once invoked, about $0.0002 per session on Opus 5. A static security scan graded it A with 1 finding (makes network calls). No closer match exists in the catalogue, so it is treated as the original; first seen 2026-09-12.
Other skills, from other repositories
ecs-genai
Use whenever someone runs a GPU / ML / GenAI / LLM workload on Amazon ECS: GPU on ECS, ECS GPU-optimized AMI, g4dn/g5/g6/p4/p5 on ECS, which ECS launch type for GPU, Inferentia/Trainium/Neuron on ECS, distributed training, model inference or vLLM/Triton/TGI/Ray on ECS, Capacity Blocks, GPU sharing, ASG per GPU type.…
eks-genai
Use whenever someone is building, training, fine-tuning, or serving a generative AI / LLM workload on Amazon EKS — phrased as "GPU vs Trainium/Inferentia", "vLLM on EKS", "Ray Serve / KubeRay", "distributed training on EKS", "FSx for Lustre for ML", "Karpenter for GPU", "EFA / NCCL multi-node", "DCGM / Neuron…
serving-llms-vllm
Serves LLMs with high throughput using vLLM's PagedAttention and continuous batching. Use when deploying production LLM APIs, optimizing inference latency/throughput, or serving models with limited GPU memory. Supports OpenAI-compatible endpoints, quantization (GPTQ/AWQ/FP8), and tensor parallelism.
tensorrt-llm
High-throughput LLM inference on NVIDIA GPUs.
cli-eval
Create and run evaluation suites, watch live benchmark progress, view scorecards, compare model performance, and integrate eval runs with CI workflows from the CLI.
cli-backup-sync
Backup and restore OmniRoute data from the CLI. Trigger incremental snapshots, sync to cloud storage, manage backup schedules, and restore from archive files.