searxng-deepdive: Instructions file for Codex

AGENTS.md — guide for AI assistants editing this codebase

If you are an AI agent (Claude Code, Cursor, Cline, Aider, Continue, Codex, …) about to make changes here, read this first. It encodes what humans wish you already knew so we don't repeat the same lessons.

For human-facing docs see README.md. This file is for agent-readable working conventions.

---

What this project is, in three sentences

searxng-deepdive is an MCP server that wraps a SearXNG instance for LLM agents. It exposes four tools (search, search_on_engines, search_by_category, web_url_read) with schemas and descriptions designed for agent-tool-selection ergonomics, not just API completeness. Its design DNA is: convert every silent-wrong code path into an informatively-wrong one — empty results and wrong-tool calls must always carry a hint specific enough that a retry has a real chance.

---

Setup

Node ≥20.18.1. From repo root:

npm install
npm run build      # tsc — strict mode, noUnusedLocals/Parameters
npm test           # vitest — should be 100+ tests, ~4s

Live development against a running SearXNG:

SEARXNG_URL=http://127.0.0.1:7979 npm run probe   # exercise the SearxngClient directly
SEARXNG_URL=http://127.0.0.1:7979 npm run dev     # start the MCP stdio server

probe bypasses MCP entirely and prints search results — fastest dev loop when you want to see what SearXNG is actually returning.

---

File map

src/
├── index.ts          # stdio MCP entrypoint; probes /config at startup
├── searxng.ts        # SearXNG HTTP client (getConfig, search, searchMultiPage)
├── schemas.ts        # Zod schemas for tool inputs (single source of truth
│                     #   for runtime validation + JSON schema sent to MCP)
├── descriptions.ts   # Tool description builders, generated dynamically from
│                     #   the live SearxngConfig at server startup
├── tools.ts          # Tool registration + dispatch; doSearch unifies the
│                     #   three search variants. Exports normalizeName,
│                     #   validateEngineSelection, validateCategorySelection,
│                     #   buildZeroResultHint, trimToCompact for tests.
└── url-reader.ts     # web_url_read implementation: undici fetch +
                      #   content-type-aware conversion (HTML/XML →
                      #   node-html-markdown; JSON → pretty-printed code
                      #   fence; plaintext → passthrough) + extraction
                      #   modes (readHeadings, section, paragraphRange,
                      #   window) + 10 MB stream-cap on the body

tests/
├── normalize-name.test.ts        # case/whitespace normalization
├── validators.test.ts            # engine/category cross-validation w/ hints
├── zero-result-hint.test.ts      # every hint trigger + its inverse
├── trim-to-compact.test.ts       # response trimming
├── descriptions.test.ts          # ANTI-PATTERN regex checks for description copy
├── searxng-client.test.ts        # HTTP client w/ undici MockAgent
└── url-reader.test.ts            # extraction helpers + HTTP integration

scripts/
└── probe.ts          # dev-time SearxngClient exerciser (not in published bundle)

.github/workflows/
├── test.yml          # push/PR/dispatch — Node 20/22/24 × {ubuntu,macos,windows} matrix
└── publish.yml       # tags v*.*.* — verifies version, runs tests, then dual-publishes:
                      #   1. npm tarball with sigstore provenance (npm Trusted Publishing)
                      #   2. registry.modelcontextprotocol.io entry (mcp-publisher OIDC).
                      #   server.json version fields are rewritten from the tag at run
                      #   time so contributors only edit package.json per release.