vector-search

MoFlo Vector Search (RAG)

Semantic search over your own documents, backed by moflo's HNSW index in .moflo/moflo.db (node:sqlite, Node 22+ built-in). Small enough to ship in a devDependency; fast enough for interactive retrieval at 100k–1M vectors.

When to Use This vs memory-patterns

• memory-patterns — structured, namespaced memory you own (sessions, learnings, patterns). Keys matter. Entries are conceptual units.
• This skill (vector-search) — search over documents you've ingested for retrieval. Entries are content chunks. Keys are just stable IDs for dedupe.

Both use the same index; the difference is how you chunk and what you put in the value field.

Ingest

Two paths. Pick the one that matches your source of truth:

A. Ad-hoc ingest from Claude Code

for (const [id, text] of chunks) {
  await mcp.memory_store({
    namespace: 'docs',        // your RAG corpus
    key: id,                  // stable ID for this chunk (file path + offset, etc.)
    value: text,              // the chunk content — what gets embedded
    tags: ['doc', docType],
    upsert: true,
  });
}

B. Repeatable ingest from the filesystem

Use moflo's shipped indexers (cross-platform, skip unchanged chunks via a hash file):

# Guidance docs → namespace 'guidance'
node .claude/scripts/index-guidance.mjs

# Code structure → namespace 'code-map'
node .claude/scripts/generate-code-map.mjs

# Your own corpus — write a small indexer that loops over files and calls
# memory_store. Model it on bin/index-patterns.mjs.

Retrieve

const hits = await mcp.memory_search({
  namespace: 'docs',
  query: userQuestion,
  limit: 8,
  threshold: 0.35,   // drop irrelevant noise; tune per-corpus
});

// hits[] = [{ key, namespace, value, similarity, ... }]

Plug the retrieved value strings into your prompt as context. Filter or rerank higher up if needed.

Chunking Rules That Actually Matter

1. One idea per chunk. 200–800 tokens. Smaller for code, larger for prose.
2. Include enough context for the chunk to stand alone. A bare "it does X" chunk won't retrieve well because the embedding has no subject.
3. Deduplicate by stable ID. Re-ingest is a no-op if key + value hash match. Cheap to re-run.
4. Keep metadata minimal in value. The embedding is built from the full value — noisy prefixes dilute the vector. Put structured metadata in tags or a separate meta:${id} key.