Getting it into your agent
One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.
npx agentmods add skills/davidtoby/agent-skills/vllmnpx skills add davidtoby/agent-skills --skill vllmgit clone --depth 1 https://github.com/davidtoby/agent-skillsWrote this? Show the measurements
A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.
[](https://agentmods.dev/skills/davidtoby/agent-skills/vllm)<a href="https://agentmods.dev/skills/davidtoby/agent-skills/vllm"><img src="https://agentmods.dev/badge/skills/davidtoby/agent-skills/vllm.svg" alt="Measured on agentmods" height="20"></a>What it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5 | $0.00027 | $0.02506 |
| Opus 5 | $0.00014 | $0.01253 |
| Sonnet 5 | $0.00005 | $0.00501 |
| Haiku 4.5 | $0.00003 | $0.00251 |
Grade A, and why
serving-llms-vllm scanned grade A with 1 finding against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured yesterday.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Makes network callslowCapability
Not a fault in itself. Listed so you know the mod talks to something, and to what.
curl http://localhost:9090/metrics | grep vllm This is a copy
89% identical to serving-llms-vllm — 34 lines differ, which has more behind it and is treated as the original. This page carries a canonical link to it rather than competing with it.
How it starts
The opening of the file, as written. The whole thing — 372 lines — stays where its author put it; the contents beside it link to each section on GitHub.
vLLM - High-Performance LLM Serving
When to use
Use when deploying production LLM APIs, optimizing inference latency/throughput, or serving models with limited GPU memory. Supports OpenAI-compatible endpoints, quantization (GPTQ/AWQ/FP8), and tensor parallelism.
Quick start
vLLM achieves 24x higher throughput than standard transformers through PagedAttention (block-based KV cache) and continuous batching (mixing prefill/decode requests).
Installation:
pip install vllm
Basic offline inference:
from vllm import LLM, SamplingParams
llm = LLM(model="meta-llama/Llama-3-8B-Instruct")
sampling = SamplingParams(temperature=0.7, max_tokens=256)
outputs = llm.generate(["Explain quantum computing"], sampling)
print(outputs[0].outputs[0].text)
OpenAI-compatible server:
vllm serve meta-llama/Llama-3-8B-Instruct
# Query with OpenAI SDK
python -c "
from openai import OpenAI
client = OpenAI(base_url='http://localhost:8000/v1', api_key='EMPTY')
print(client.chat.completions.create(
model='meta-llama/Llama-3-8B-Instruct',
messages=[{'role': 'user', 'content': 'Hello!'}]
).choices[0].message.content)
"
Common workflows
Workflow 1: Production API deployment
Copy this checklist and track progress:
Deployment Progress:
- [ ] Step 1: Configure server settings
- [ ] Step 2: Test with limited traffic
- [ ] Step 3: Enable monitoring
- [ ] Step 4: Deploy to production
- [ ] Step 5: Verify performance metrics
Step 1: Configure server settings
Choose configuration based on your model size:
# For 7B-13B models on single GPU
vllm serve meta-llama/Llama-3-8B-Instruct \
--gpu-memory-utilization 0.9 \
--max-model-len 8192 \
--port 8000
# For 30B-70B models with tensor parallelism
vllm serve meta-llama/Llama-2-70b-hf \
--tensor-parallel-size 4 \
--gpu-memory-utilization 0.9 \
--quantization awq \
--port 8000
# For production with caching and metrics
vllm serve meta-llama/Llama-3-8B-Instruct \
--gpu-memory-utilization 0.9 \
--enable-prefix-caching \
--enable-metrics \
--metrics-port 9090 \
--port 8000 \
--host 0.0.0.0
What ships with it
4 files beside SKILL.md in the same directory: the scripts, references and assets a skill reads on demand. Not counted in the per-session cost; read them before you install if any of them is executable.
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- yesterday First seen · 372 lines · 27 tokens per session scan A dbc02611b59f
serving-llms-vllm is a skill published in the GitHub repository davidtoby/agent-skills (10 stars, last pushed 1mo ago), licensed MIT. It adds 27 tokens to every session and 2,506 once invoked, about $0.0001 per session on Opus 5. A static security scan graded it A with 1 finding (makes network calls). It is 89% identical to serving-llms-vllm, differing in 34 lines, and is treated as a copy.
Other skills, from other repositories
ehr-analysis
End-to-end EHR predictive modeling pipeline with PyHealth, covering dataset loading, task definition, model training, evaluation, calibration, and clinical interpretation.
bindcraft
End-to-end binder design using BindCraft hallucination. Use this skill when: (1) Designing protein binders with built-in AF2 validation, (2) Running production-quality binder campaigns, (3) Using different design protocols (fast, default, slow), (4) Need joint backbone and sequence optimization, (5) Want high…
esm2-sequence-scoring
ESM2 protein language model for sequence scoring, embeddings, and plausibility checks. Use this skill when: (1) Computing pseudo-log-likelihood (PLL) scores, (2) Getting protein embeddings for clustering, (3) Filtering designs by sequence plausibility, (4) Zero-shot variant effect prediction, (5) Analyzing…
scrna-preprocessing-clustering
Standard scRNA-seq preprocessing and clustering with Scanpy. Use for QC, normalization, HVG selection, PCA, neighbor graph construction, UMAP, Leiden clustering, and export of an analysis-ready AnnData object.
alignment-and-mapping
Workflow for read alignment, sorting, indexing, mapping statistics, and downstream-ready alignment artifacts.
machine-learning-for-omics
Workflow for predictive modeling, biomarker discovery, survival modeling, and explainability over omics-derived features.