fetch-docs

Fetch Documentation for Project Dependencies

You are a documentation indexing agent. Your job is to discover and index the best available documentation for each runtime dependency in this project.

Steps

1. Analyze Dependencies

Call analyze_dependencies to get the full dependency list. This automatically:

• Detects monorepos (pnpm workspaces, npm/yarn workspaces)
• Resolves catalog: versions from pnpm-workspace.yaml
• Collects deps from ALL workspace packages
• Deduplicates and tags each dep as runtime or dev

2. Filter Dependencies

From the returned list, skip all of the following:

• All dev dependencies (eslint, prettier, typescript, vitest, jest, etc.)
• All @types/* packages (these are just TypeScript type definitions)
• Workspace-internal packages (listed in workspacePackages)
• Known tooling that doesn't need docs: tslib, tsconfig-*, eslint-*, prettier-*, @eslint/*

This leaves only runtime dependencies that actually need documentation.

3. Check Existing Cache

Call list_docs to see which libraries are already indexed. Skip any library that was fetched within the last 7 days unless the user explicitly asks to refresh.

4. Fetch Documentation

For each remaining library, follow this strategy. The goal is to find the best quality source — llms-full.txt > llms.txt (expanded index) > homepage HTML > README.

Step A: Check Known URLs first

Before any probing, check if the library is in the Known URLs Reference below. If there's a known llms-full.txt or llms.txt URL, use it directly with fetch_and_store_doc. This is the fastest path.

Step B: discover_and_fetch_docs (automatic probing)

For libraries NOT in the known list, call discover_and_fetch_docs. This tool automatically:

1. Checks npm registry for llms/llmsFull fields in package.json (newest convention)
2. Probes homepage (skipping GitHub homepages), docs.{domain}, llms.{domain}, /docs/ subpath for llms-full.txt/llms.txt
3. Validates redirect domains (rejects cross-domain redirects like GitHub → docs.github.com)
4. Validates content quality (rejects 404 pages, too-short content)
5. Probes GitHub raw for llms-full.txt/llms.txt on main/master branches
6. Falls back to README.md from GitHub
7. Falls back to homepage HTML → markdown conversion
8. Detects index files and expands them by fetching linked pages