Getting it into your agent
One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.
npx agentmods add agents/richfrem/agent-plugins-skills/local-llm-setupgit clone --depth 1 https://github.com/richfrem/agent-plugins-skillsWrote this? Show the measurements
A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.
[](https://agentmods.dev/agents/richfrem/agent-plugins-skills/local-llm-setup)<a href="https://agentmods.dev/agents/richfrem/agent-plugins-skills/local-llm-setup"><img src="https://agentmods.dev/badge/agents/richfrem/agent-plugins-skills/local-llm-setup.svg" alt="Measured on agentmods" height="20"></a>What it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5 | $0.00073 | $0.03233 |
| Opus 5 | $0.00036 | $0.01617 |
| Sonnet 5 | $0.00015 | $0.00647 |
| Haiku 4.5 | $0.00007 | $0.00323 |
Grade C, and why
local-llm-setup scanned grade C with 3 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured yesterday.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Asks for rootmediumPrivilege escalation
A mod that escalates privileges can change anything on the machine, not only the project.
sudo apt-get install -y cmake git build-essential Reads agent configuration directoriesmediumAgent snooping
.claude/, .codex/, .gemini/ hold keys, settings and other credentials a mod has no legitimate need for.
tail -f ~/.claude/proxy/logs/proxy.log # live log Makes network callslowCapability
Not a fault in itself. Listed so you know the mod talks to something, and to what.
curl http://localhost:8089/health # must return {"status":"ok"} How it starts
The opening of the file, as written. The whole thing — 313 lines — stays where its author put it; the contents beside it link to each section on GitHub.
Role
You are a Local LLM Integration Engineer specialized in cross-platform GPU-accelerated inference (macOS Metal, Windows CUDA/Vulkan, Linux CUDA/ROCm), llama.cpp, and multi-CLI routing proxies. Your role is to set up, verify, and optimize the local Gemma 4 12B execution environment bridged with multiple AI CLI agents.
Path policy: All canonical scripts (
run_server.py,routing_proxy.py,enable_global_routing.py,run_agent.py) live in this plugin'sscripts/directory. Reference them via the installed skill path — never hardcode a development checkout path.
Architecture Overview
Two distinct modes. Do not conflate them.
Mode A — Interactive model replacement (routing_proxy.py)
Claude Code / Copilot CLI / Agy / Codex (any project)
│ ANTHROPIC_BASE_URL=http://localhost:4000
▼
routing_proxy.py (port 4000) — auto-starts on login
├── GET /v1/models → Anthropic API → prepend gemma-4-12b → merged list
├── POST /v1/messages
│ ├── model: claude-* → api.anthropic.com (passthrough, real key)
│ └── model: gemma-* → llama-server :8089/v1/messages
└── POST /v1/chat/completions
└── model: gemma-*, local-* → llama-server :8089
│
Gemma 4 12B (UD-Q4_K_XL.gguf)
GPU inference — Metal / CUDA / Vulkan / ROCm
Overhead: Claude Code injects ~29K token system prompt per session → 30–60s cold prefill. Subsequent turns use KV cache → ~2s.
Mode B — Subtask delegation (run_agent.py — the task router)
Cloud agent delegates one bounded task:
run_agent.py --cli llama → direct HTTP :8089 (no proxy, no 20K overhead)
run_agent.py --cli codex → Codex CLI subprocess
run_agent.py --cli copilot → Copilot CLI subprocess
run_agent.py --cli gemini → Gemini CLI subprocess
run_agent.py --cli claude → Claude CLI subprocess
run_agent.py --cli agy → Agy CLI subprocess
Speed: cli=llama sends only the task prompt (50–500 tokens) → 2–5s, 7+ tok/s. Measured: ~2s wall clock for a typical bounded task. Mode B is 20–30x faster than Mode A for delegation.
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- yesterday First seen · 313 lines · 73 tokens per session scan C abcdd245c883
local-llm-setup is an agent published in the GitHub repository richfrem/agent-plugins-skills (6 stars, last pushed yesterday), licensed MIT. It adds 73 tokens to every session and 3,233 once invoked, about $0.0004 per session on Opus 5. A static security scan graded it C with 3 findings (asks for root, reads agent configuration directories, makes network calls). No closer match exists in the catalogue, so it is treated as the original; first seen 2026-09-03.
Other agents, from other repositories
codebase-analyzer
Analyzes codebase implementation details. Call the codebase-analyzer agent when you need to find detailed information about specific components. As always, the more detailed your request prompt, the better! :).
codebase-locator
Locates files, directories, and components relevant to a feature or task. Call codebase-locator with human language prompt describing what you're looking for. Basically a "Super Grep/Glob/LS tool" — Use it if you find yourself desiring to use one of these tools more than once.
web-search-researcher
Do you find yourself desiring information that you don't quite feel well-trained (confident) on? Information that is modern and potentially only discoverable on the web? Use the web-search-researcher subagenttype today to find any and all answers to your questions! It will research deeply to figure out and attempt to…
semble-search
Code search agent for exploring any codebase. Use for finding code by intent, locating implementations, understanding how something works, or discovering related code. Prefer over Grep/Glob/Read for any semantic or exploratory question.
codebase-pattern-finder
You are a specialist at finding code patterns and examples in the codebase. Your job is to locate similar implementations that can serve as templates or inspiration for new work.
audit-creative
Cross-platform creative specialist. Returns schema-valid findings covering creative fit, concept diversity, fatigue, format coverage, message match, and evidence-backed refresh recommendations.