local-llm-mcp: Instructions file for Claude Code

CLAUDE.md

What this is

An MCP server that answers prompts with a local model through Ollama, synchronously. Claude calls local_ask or local_classify and gets the answer back in the same turn.

This is the deliberate opposite of the sibling codex-offload server. That one exists because Codex jobs take minutes and must not block; this one exists because a local 7B answers in seconds and wrapping that in a job store, disk state and polling would be pure overhead. If a tool here would need to be polled, it belongs in codex-offload instead, not this one.

Commands

npm run build     # tsc -> dist/
npm run watch     # tsc --watch
npm test          # build, then node --test test/*.test.js

Tests run against the compiled dist/, which is why npm test builds first. They cover matchLabel — the label validation — and nothing else. The HTTP path is deliberately uncovered: exercising it needs a live model, so verify it by driving the built server over stdio with a JSON-RPC script (initialize → notifications/initialized → tools/call).

Layout

• src/index.ts — MCP server, tool definitions and their descriptions
• src/ollama.ts — HTTP client, plus the pure helpers that keep model output honest

Configuration

env	default	meaning
OLLAMA_HOST	http://127.0.0.1:11434	Ollama endpoint
LOCAL_LLM_MODEL	qwen2.5-coder:7b	default model tag
LOCAL_LLM_TIMEOUT_MS	120000	per-request timeout

Notes

Do not enable the integrated GPU. On shared-memory hardware the iGPU competes for the same system RAM, so generation — which is memory-bandwidth-bound — gets slower. Measured on a Radeon 890M with llama3.2:3b: 33.7 tok/s on CPU vs 26.2 tok/s with OLLAMA_IGPU_ENABLE=1. Prompt ingest roughly doubles (285 → 600 tok/s), so the flag is only arguable for long-input / short-output work, and never as a default. Ollama drops iGPUs by default; leave it that way.

Throughput sets the tool design. Measured on CPU: ~34 tok/s for a 3B, ~16 tok/s for a 7B, scaling cleanly with size — there is no memory cliff, since 61 GB of RAM means the constraint is bandwidth, not capacity. At 16 tok/s a 500-token answer costs ~31s, so maxTokens is the real latency lever and short outputs are the only comfortable shape.