What is searxNcrawl AGENTS.md?

Repository instructions for coding agents working on searxNcrawl, a Python package, command-line tool, and MCP server for web search and Markdown crawling. They describe the project's layout, tools, and working rules.

Use them when modifying searxNcrawl's search, crawling, command-line, or MCP server code, or when checking its project-specific commands and conventions.

They give an agent the project context it needs before editing code, reducing guesses about where features belong and which commands to use.

How DasDigitaleMomentum/searxNcrawl configures Codex and OpenCode

Reuse

Borrowing it

Nothing to install: this file belongs to DasDigitaleMomentum/searxNcrawl. Take a copy, put it at the same path in your own repository, and replace the rules that are about this project with yours.

Copy the file

curl -O https://raw.githubusercontent.com/DasDigitaleMomentum/searxNcrawl/main/AGENTS.md

Clone the repo

git clone --depth 1 https://github.com/DasDigitaleMomentum/searxNcrawl

Made for: Codex, OpenCode.

Wrote this? Show the measurements

A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.

README.md

[![agentmods](https://agentmods.dev/badge/instructions/dasdigitalemomentum/searxncrawl/agents-md/github.svg)](https://agentmods.dev/instructions/dasdigitalemomentum/searxncrawl/agents-md)

Your own site

<a href="https://agentmods.dev/instructions/dasdigitalemomentum/searxncrawl/agents-md"><img src="https://agentmods.dev/badge/instructions/dasdigitalemomentum/searxncrawl/agents-md/github.svg" alt="Measured on agentmods" height="20"></a>

Or the 80×15 button, for a site that already has a row of RSS and ATOM ones. Only the verdict fits; the numbers stay here.

Your own site · 80×15

<a href="https://agentmods.dev/instructions/dasdigitalemomentum/searxncrawl/agents-md"><img src="https://agentmods.dev/badge/instructions/dasdigitalemomentum/searxncrawl/agents-md.svg" alt="Reviewed on agentmods" width="80" height="20"></a>

Per session 1,111 This file is loaded in full into every session.

When invoked 1,111 The same file — it is already loaded in full.

Security scan A 0 findings. A grade says what 26 rules found in the file — not that it is safe.

Origin original No closer match found in the catalogue.

Token cost

What it costs to keep this loaded

Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.

Model	Per session	Once invoked
Fable 5.1	$0.01111	$0.01111
Opus 5.5	$0.00444	$0.00444
Sonnet 5.5	$0.00222	$0.00222
Haiku 4.5	$0.00111	$0.00111

Measured 8d ago against content hash 1193730fa206, method: parsed. Prices are Anthropic first-party input rates as of 2026-10-07, from the pricing page.

Security

Grade A, and why

searxNcrawl AGENTS.md scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 8d ago.

A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.

Nothing flagged

None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.

AGENTS.md · 61 lines

How it starts

The opening of the file, as written. The whole thing — 61 lines — stays where its author put it; the contents beside it link to each section on GitHub.

AGENTS.md

Guidance for coding agents (Claude Code, Codex, Cursor, Copilot and others) working in this repository. Humans should start with the README and the contributing guide.

Project

searxNcrawl is an MCP server, CLI and Python package for web search through SearXNG and/or You.com and Markdown crawling through Crawl4AI and Playwright. It is published on PyPI as searxncrawl; server.json describes it as io.github.DasDigitaleMomentum/searxncrawl for the MCP Registry.

Layout

Path	Contents
`crawler/mcp_server.py`	FastMCP server with the `crawl`, `crawl_site` and `search` tools; console scripts `searxncrawl` and `crawl-mcp`
`crawler/cli.py`	`crawl`, `search` and `crawl-capture` commands
`crawler/__init__.py`	Public Python API (`crawl_page`, `crawl_site` and async variants)
`crawler/config.py`	Crawl4AI run configuration and overrides
`crawler/search_orchestrator.py`	Search providers (SearXNG, You.com), parallel queries and result merging
`crawler/site.py`	Multi-page site crawling
`crawler/markdown_dedup.py`	Removal of repeated Markdown blocks
`crawler/document.py`, `builder.py`, `references.py`	`CrawledDocument` model and Markdown/JSON output
`crawler/auth.py`, `session_capture.py`	Authenticated crawling and session capture
`crawler/browser_setup.py`	Automatic Chromium install
`crawler/setup.py`	`searxncrawl setup`: registers the server with installed coding agents
`tests/`	pytest suite
`docs/usage/`	User guides linked from the README
`docs/modules/`, `docs/features/`	Architecture notes per module and feature
`server.json`	MCP Registry metadata
`mcpb/`	Claude Desktop MCPB source and legacy Smithery compatibility builder
`deploy/arcade/`	Optional Arcade adapter with separate dependencies and container; standard installations use the native MCP server

Read the full file on GitHub · 61 lines

Changes

What this file has done since we first saw it

Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.

8d ago Changed · +5 lines · +121 tokens per session 1193730fa206
21d ago First seen · 56 lines · 990 tokens per session scan A 22ad6a711aa9

Subscribe to this mod's changes

searxNcrawl AGENTS.md is an instructions file published in the GitHub repository DasDigitaleMomentum/searxNcrawl (185 stars, last pushed yesterday), licensed MIT. It adds 1,111 tokens to every session, about $0.0044 per session on Opus 5.5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-10-16.

Building agents Python crawl4ai llm-tools mcp mcp-server searxng

searxNcrawl AGENTS.md next to 6 instructions from other repositories
Mod	Type	Stars	Last push	Licence	Scan	Tokens	Works with
searxNcrawl AGENTS.mdDasDigitaleMomentum/searxNcrawl	Instructions file	185	yesterday	MIT	A	1,111	Codex, OpenCode
vscode buildNext.instructions.mdmicrosoft/vscode	Instructions file	195k	yesterday	MIT	A	9,719	GitHub Copilot
next.js AGENTS.mdvercel/next.js	Instructions file	143k	today	MIT	A	7,461	Codex, OpenCode
langchain AGENTS.mdlangchain-ai/langchain	Instructions file	149k	yesterday	MIT	A	4,680	Codex, OpenCode
vscode oss-third-party-notices.instructions.mdmicrosoft/vscode	Instructions file	195k	yesterday	MIT	A	5,516	GitHub Copilot
vscode copilot-instructions.mdmicrosoft/vscode	Instructions file	195k	yesterday	MIT	A	3,093	GitHub Copilot
deepseek-harness AGENTS.mddeepseek-ai/deepseek-harness	Instructions file	274k	2d ago	MIT	A	4,313	Codex, OpenCode

searxNcrawl: Instructions file for Codex