ingest

A command that incrementally indexes Markdown files and refreshes their search index, links, topics, and derived information. Markdown is a plain-text format commonly used for documentation.

In plain words
What is it for?
Use it to add or update documentation in the index, rebuild embeddings when needed, process one file, reindex everything, or preview an ingestion run.
Why use it?
It updates only new, changed, or deleted files, avoiding unnecessary processing. Preview mode can estimate the work without contacting providers or writing changes.

Command

Install

Getting it into your agent

One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.

agentmods
npx agentmods add commands/geckse/markdown-vdb/ingest
Clone the repo
git clone --depth 1 https://github.com/geckse/markdown-vdb
Per session 16 Only the description is in the session, so the agent can decide to use it. The body loads when it is invoked.
When invoked 1,421 The whole file, excluding the scripts and references it only reads on demand.
Security scan A 0 findings. Scan, not verified.
Origin original No closer match found in the catalogue.
Token cost

What it costs to keep this loaded

Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.

ModelPer sessionOnce invoked
Fable 5 $0.00016 $0.01421
Opus 5 $0.00008 $0.00711
Sonnet 5 $0.00003 $0.00284
Haiku 4.5 $0.00002 $0.00142

Measured 3d ago against content hash dc4660e91f95, method: parsed. Prices are Anthropic first-party input rates as of 2026-08-30, from the pricing page.

Security

Grade A, and why

ingest scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 3d ago.

A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.

Nothing flagged

None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.

docs/cli/commands/ingest.md · 140 lines

How it starts

The opening of the file, as written. The whole thing — 140 lines — stays where its author put it; the contents beside it link to each section on GitHub.

mdvdb ingest

Discover and index Markdown files. Normal ingestion is incremental: unchanged files are skipped by content hash, while new, changed, and deleted files update the vector index, lexical index, link graph, schemas, analysis, and computed fields.

Usage

mdvdb ingest [OPTIONS]

Options

Flag Default Description
--reindex false Re-embed all files, ignoring stored content hashes
--file <PATH> Ingest one project-relative Markdown file
--preview false Parse and estimate the operation without provider calls or writes
--json-lines false Stream progress and the final result as newline-delimited JSON

--full remains a hidden, deprecated alias for --reindex. The command also accepts all global options.

When to reindex

Use --reindex after changing the embedding provider, model, or dimensions. mdvdb also reports embedding incompatibility through mdvdb status.

mdvdb ingest --reindex

Single-file and preview modes

# Update one file
mdvdb ingest --file docs/getting-started.md

# Re-embed one file
mdvdb ingest --file docs/getting-started.md --reindex

# Estimate files, chunks, tokens, and calls without mutating anything
mdvdb ingest --preview --json

Preview output contains per-file New, Changed, or Unchanged status plus aggregate chunk, token, and API-call estimates.

What ingestion updates

At a high level, mdvdb:

  1. Discovers files using source and ignore configuration.
  2. Parses frontmatter and Markdown structure, chunks content, and computes hashes.
  3. Embeds new or changed chunks and semantic edge context, with provider-aware batching and input limits.
  4. Reconciles the vector snapshot and Tantivy lexical index, deleted files, and the link/relation graph.
  5. Refreshes automatic communities (Leiden by default, with K-means available), user Topics, and Shard-local topic sidecars.
  6. Rebuilds global and scoped schemas on a collection-wide ingest.
  7. Runs Formula first, then Lookup/Rollup, materializing declared computed keys in frontmatter.
  8. Atomically persists coherent index/module state and commits the lexical index.

Read the full file on GitHub · 140 lines

Changes

What this file has done since we first saw it

Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.

  1. 3d ago First seen · 140 lines · 16 tokens per session scan A dc4660e91f95

Subscribe to this mod's changes

ingest is a command published in the GitHub repository geckse/markdown-vdb (23 stars, last pushed 19d ago), licensed MIT. It adds 16 tokens to every session and 1,421 once invoked, about $0.0001 per session on Opus 5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-30.