web-search

Web Search

Use this when a small local LLM needs facts from multiple pages and raw fetch results would overflow context.

For crawly-mcp, prefer the built-in bounded path first:

• read status and next_action before concluding anything. status="no_results" means a provider searched and genuinely found nothing, so reword the query. status="blocked", "unavailable", "timeout", or "navigation_failed" means providers could not answer: that is not evidence that the topic is empty, so call again with a suggested_providers entry, ask providers what is usable, or report the failure accurately. Never tell the user a search found nothing when the status says a provider failed.
• search cascades providers on its own and reports provider (the winner) plus attempted (what each provider did). Do not re-run the same query unchanged.
• call page_search(url, query) first when the task is really "find this within a known page or docs site"
• call fetch(urls=[...], content_format="text") instead of raw HTML
• keep urls small, usually 1..3 per fetch for local models
• answer in concise prose by default
• switch to schema-first extraction only when the task is large enough that direct prose synthesis would overflow context or when the user explicitly asks for structured output
• if schema extraction is needed, reduce extracted records in batches, not all at once

When To Use

• search + fetch + synthesize across multiple pages
• local or small-context LLMs processing crawled pages
• context overflows after a few fetches
• evidence gathering where you need structured per-source notes

Do not use for single-page questions or when search snippets already answer the question. For a single known page or docs entrypoint, use page_search(...) before broader web search.

Default Workflow

1. Start with page_search(url, query) when you already know the site or docs entrypoint.
2. Otherwise run search(...) and keep only the most relevant URLs.
3. Fetch with content_format="text" and keep batches small.
4. Answer in concise prose when the source set is small and clear.
5. Use per-source structured notes only when the task is broad enough to need batching.
6. If structured notes are needed, reduce in small batches, then merge rollups if needed.
7. Keep any intermediate notes until the answer is verified.