retrieval

Retrieval

Search has two backends. One setting, agents.retrieval in src/docs.config.ts, chooses between them. The first backend needs nothing. The second needs an account, and it adds semantic matching.

lexical: the default, and the floor

This backend scores the build's own search index inside the Worker. It needs no account setup, no provisioning, and no external call, so it works on the first deploy. It weights term frequency towards titles and headings. Two corrections matter more than the weights themselves. First, the score damps occurrence counts by section length, so a long API reference cannot outscore a short, on-topic paragraph. Second, it measures coverage across the whole page, because a question's terms are usually spread between a page's title and its body.

The backend also does light suffix stripping. Without it, "deployment" fails to match a page titled "Deploy your site", and the query lands on whichever page repeats the exact word most.

This backend works well for a few hundred pages, which covers most documentation sites.

Two quality filters

Whichever backend runs, worker/agent/retrieval.ts filters the results before grouping them into pages. Two constants control the filter. Both were calibrated by measurement, and both are specific to this corpus:

Constant	Value	What it cuts
MIN_VECTOR_SIMILARITY	0.54	vector-only hits whose cosine similarity says the passage is about something else
MIN_KEYWORD_SCORE	12	keyword hits that matched on one incidental common word rather than a real term

The retrieval logic keeps a passage if it clears either bar. This matters, because an exact identifier, such as a header name, a CNAME record, or an error code, often scores zero on vectors. A paraphrased question, in turn, often has no keyword match at all.