Getting it into your agent
One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.
npx skills add CrestApps/CrestApps.AgentSkills --skill orchardcore-ai-web-crawlersgit clone --depth 1 https://github.com/CrestApps/CrestApps.AgentSkillsWrote this? Show the measurements
A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.
[](https://agentmods.dev/skills/crestapps/crestapps.agentskills/orchardcore-ai-web-crawlers)<a href="https://agentmods.dev/skills/crestapps/crestapps.agentskills/orchardcore-ai-web-crawlers"><img src="https://agentmods.dev/badge/skills/crestapps/crestapps.agentskills/orchardcore-ai-web-crawlers/github.svg" alt="Measured on agentmods" height="20"></a>Or the 80×15 button, for a site that already has a row of RSS and ATOM ones. Only the verdict fits; the numbers stay here.
<a href="https://agentmods.dev/skills/crestapps/crestapps.agentskills/orchardcore-ai-web-crawlers"><img src="https://agentmods.dev/badge/skills/crestapps/crestapps.agentskills/orchardcore-ai-web-crawlers.svg" alt="Reviewed on agentmods" width="80" height="20"></a>What it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5.1 | $0.00204 | $0.02136 |
| Opus 5.5 | $0.00082 | $0.00854 |
| Sonnet 5.5 | $0.00041 | $0.00427 |
| Haiku 4.5 | $0.00020 | $0.00214 |
Grade A, and why
orchardcore-ai-web-crawlers scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured today.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Nothing flagged
None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.
How it starts
The opening of the file, as written. The whole thing — 143 lines — stays where its author put it; the contents beside it link to each section on GitHub.
Orchard Core AI Web Crawlers
Scrape public websites into the AI knowledge base
You are an Orchard Core expert. Generate configuration and code for CrestApps web crawlers: a Web AI data source is a knowledge base target, and one or more web crawlers point at it. Each crawler scrapes one site with a crawl strategy, cleans each page to text, and indexes it with its URL kept for citations.
Guidelines
- Enable
CrestApps.OrchardCore.AI.WebCrawlers; it depends onCrestApps.OrchardCore.AI.DataSources, which comes along. - The Web data source needs a knowledge base index with an embedding deployment that declares
textEmbedding(seeorchardcore-ai-data-sources). It has no connection settings and no field mapping: each page is one document keyed by its URL. - Sitemap is the built-in strategy. A crawler needs a base URL or an explicit sitemap URL.
- Crawl politely: keep Max concurrent requests low, set a meaningful User-Agent, and use include and exclude patterns to stay inside the content you need.
- Only crawl sites you are allowed to scrape; treat scraped text as untrusted input to the model.
- Install the package in the web/startup project.
Setup
- Create a knowledge base index under Search > Indexing with an embedding deployment.
- Under Artificial Intelligence > Data Sources, choose Add Data Source, pick Web, and select the destination index.
- Under Artificial Intelligence > Web Crawlers, choose Add Web Crawler, pick Sitemap, and fill the fields below.
- Use Synchronize now in the row's Actions menu, or wait for the hourly task.
- Attach the Web data source to an AI profile or chat interaction on its Knowledge tab.
A recipe can create the Web data source ("Source": "Web" in the AIDataSource step, see orchardcore-ai-data-sources). Crawlers have no recipe or deployment step; create them in the admin or through ISourceCatalogManager<WebCrawler>.
Shared crawler fields
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- today First seen · 143 lines · 204 tokens per session scan A 9e4db96e4644
orchardcore-ai-web-crawlers is a skill published in the GitHub repository CrestApps/CrestApps.AgentSkills (13 stars, last pushed today), licensed MIT. It adds 204 tokens to every session and 2,136 once invoked, about $0.0008 per session on Opus 5.5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-10-22.
Other skills, from other repositories
microsoft-extensions-ai
Build provider-agnostic .NET AI integrations with Microsoft.Extensions.AI, IChatClient, embeddings, middleware, structured output, vector search, and evaluation. USE FOR: building or reviewing .NET code that uses Microsoft.Extensions.AI, Microsoft.Extensions.AI.Abstractions, IChatClient, IEmbeddingGenerator…
local-rag-search
Efficiently perform web searches using the mcp-local-rag server with semantic similarity ranking. Use this skill when you need to search the web for current information, research topics across multiple sources, or gather context from the internet without using external APIs. This skill teaches effective use of…
vector-backend-operations
Select and connect the right vector-store backend for the vector-mcp MCP server — chromadb, postgres/pgvector, qdrant, couchbase, or mongodb — and supply the correct dbtype/connection parameters that every collection and search call needs. Use when the agent must decide which engine to target, wire up…
vector-hybrid-search
Retrieve knowledge from a vector-store collection via the vector-mcp MCP server's vectorsearch tool — semantic (ANN) search, lexical BM25 search, or a hybrid of the two fused with Reciprocal Rank Fusion. Use when the agent must answer a question from an indexed corpus, pull top-k relevant chunks for RAG context, or…
databricks-genai-agent-engineering
Use this skill to review generative-AI agent design on Databricks: Mosaic AI Agent Framework and ResponsesAgent interface, Databricks AI Search index variant and sync-mode choice, retrieval and context engineering, MCP server category and trust boundaries, external model-provider selection, and Unity AI Gateway…
setup
Configure the Qdrant Power after installation. Use this skill for missing uvx, missing environment variables, unapproved environment variables, unavailable Qdrant tools, "Failed to connect" errors, and setup requests.