agentsop-llm-engine-selection

A decision guide for choosing an engine that runs large language models on your own hardware or servers. It compares options such as vLLM, SGLang, TensorRT-LLM, TGI, llama.cpp, Ollama, and MLX based on hardware, workload, and constraints.

In plain words
What is it for?
Selecting, defending, mixing, migrating, or auditing an LLM serving stack for production, batch jobs, edge devices, laptops, multi-tenant services, or structured-output systems.
Why use it?
Choosing only by a headline speed claim can lead to a poor fit for the hardware or the way the system will be used.

Skill for Claude CodeCodex

Install

Getting it into your agent

One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.

agentmods
npx agentmods add skills/agentsope/skillalchemy/agentsop-llm-engine-selection
Any agent
npx skills add agentsope/SkillAlchemy --skill agentsop-llm-engine-selection
Clone the repo
git clone --depth 1 https://github.com/agentsope/SkillAlchemy

Made for: Claude Code, Codex.

Per session 86 Skills are progressive disclosure: only the name and description are preloaded; the body loads when the skill is used.
When invoked 6,819 The whole file, excluding the scripts and references it only reads on demand.
Security scan A 0 findings. Scan, not verified.
Origin original No closer match found in the catalogue.
Token cost

What it costs to keep this loaded

Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.

ModelPer sessionOnce invoked
Fable 5 $0.00086 $0.06819
Opus 5 $0.00043 $0.03410
Sonnet 5 $0.00017 $0.01364
Haiku 4.5 $0.00009 $0.00682

Measured yesterday against content hash 4b7866571be9, method: parsed. Prices are Anthropic first-party input rates as of 2026-08-30, from the pricing page.

Security

Grade A, and why

agentsop-llm-engine-selection scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured yesterday.

A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.

Nothing flagged

None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.

skills/agentsop-llm-engine-selection/SKILL.md · 377 lines

How it starts

The opening of the file, as written. The whole thing — 377 lines — stays where its author put it; the contents beside it link to each section on GitHub.

LLM Engine Selection SOP

State-of-the-art warning. This skill is dated May 2026. The inference-engine landscape moves in 3–6 month cycles (TGI exited maintenance into deprecation in late 2025; SGLang's RadixAttention regressed vLLM's lead in 2024; TensorRT-LLM dropped its proprietary-engine requirement in 2025; Ollama added concurrent-request support mid-2025). Re-verify before betting a quarter of eng budget on any choice below.


1. 何时激活 (When to activate)

Activate this skill any time a coder-agent must:

  • Pick a serving stack for a new project (production hosting / batch / edge / dev laptop / multi-tenant SaaS / structured-output service).
  • Defend an existing stack against a "let's switch to X" pressure.
  • Migrate: justify or block a swap (e.g. TGI → vLLM, Ollama → vLLM, vLLM → TensorRT-LLM).
  • Mix: design a multi-tier deployment (e.g. premium tier on TensorRT-LLM, free tier on vLLM-AWQ, dev on Ollama).
  • Audit a recommendation that smells like benchmark-cherry-picking ("X is 5× faster").

Do not activate for:

  • Tuning a single chosen engine — defer to the dedicated skill (vllm, sglang, tensorrt-llm, llama-cpp).
  • Training/fine-tuning runtime selection — different problem class (accelerate, deepspeed, axolotl).
  • Hosted-API procurement (OpenAI / Anthropic / Bedrock) — engine choice doesn't apply.

2. 核心心智模型 (Core Mental Model)

Engine choice is a function of (hardware × workload × constraint), not "which is fastest".

There is no global ranking. Every "X beats Y by N%" headline holds only inside an unstated (hardware, batch size, ISL/OSL, model, quantization, concurrency) tuple. Change any axis and the ranking flips.

2.1 The four-axis decision space

  1. Hardware axis — NVIDIA H100/A100 (NVLink) ≠ NVIDIA L40S/RTX (PCIe-only) ≠ AMD MI300 ≠ Apple Silicon ≠ CPU-only. The interconnect topology matters as much as raw FLOPS [spheron.network 2026]. PCIe-only tensor parallelism collapses; NVLink rescues it.
  2. Workload axis — production multi-user (throughput) vs latency-bound single-stream vs offline batch vs edge single-user vs structured-output service vs multi-LoRA SaaS. Each has a different winner.
  3. Constraint axis — license (Apache vs proprietary), vendor lock-in tolerance, engineering budget (1 day vs 2 weeks setup), commercial-use clauses, on-prem vs cloud, P50 vs P99 SLA.
  4. Maturity axis — the engine's coverage of YOUR model family. A 2025-launched MoE may run on vLLM day-1 but need a 3-month wait for TensorRT-LLM, and may never get a stable GGUF.

Read the full file on GitHub · 377 lines

Files

What ships with it

4 files beside SKILL.md in the same directory: the scripts, references and assets a skill reads on demand. Not counted in the per-session cost; read them before you install if any of them is executable.

Changes

What this file has done since we first saw it

Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.

  1. yesterday First seen · 377 lines · 86 tokens per session scan A 4b7866571be9

Subscribe to this mod's changes

agentsop-llm-engine-selection is a skill published in the GitHub repository agentsope/SkillAlchemy (342 stars, last pushed 7d ago), licensed MIT. It adds 86 tokens to every session and 6,819 once invoked, about $0.0004 per session on Opus 5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-30.

Related

Other skills, from other repositories

shellgames

Play board games on ShellGames.ai — Chess, Poker, Ludo, Tycoon, Memory, and Spymaster. Use when the agent wants to play games against humans or other AI agents, join tournaments, chat with players, check leaderboards, or manage a ShellGames account. Triggers on "play chess/poker/ludo/memory", "shellgames", "join…

MemTensor/skills-vote · 103 tokens

curl-search

Web search using curl + multiple search engines (Baidu, Google, Bing, DuckDuckGo). Activates when user asks to search, look up, or query something online. Includes security enhancements: input sanitization, command injection protection, and URL encoding.

MemTensor/skills-vote · 55 tokens

skills-vote-local

Use when retrieving the most relevant skills from a local or private skill library instead of relying on network-based skill discovery.

MemTensor/skills-vote · 28 tokens

skills-vote

Find the most relevant external agent skills for the current task, then submit grounded feedback about which skills were actually used and useful in the same session. Whenever you start a task, use this skill first.

MemTensor/skills-vote · 44 tokens

pdf

Use this skill whenever the user wants to do anything with PDF files. This includes reading or extracting text/tables from PDFs, combining or merging multiple PDFs into one, splitting PDFs apart, rotating pages, adding watermarks, creating new PDFs, filling PDF forms, encrypting/decrypting PDFs, extracting images, and…

MemTensor/skills-vote · 92 tokens

xlsx

Use this skill any time a spreadsheet file is the primary input or output. This means any task where the user wants to: open, read, edit, or fix an existing .xlsx, .xlsm, .csv, or .tsv file (e.g., adding columns, computing formulas, formatting, charting, cleaning messy data); create a new spreadsheet from scratch or…

MemTensor/skills-vote · 201 tokens