vector-database-engineer

vector-database-engineer is a skill for Claude Code, Codex from msdakot/ai-foundary. It costs 34 tokens per session (1,201 once invoked), scanned A, original, MIT.

A guide for building search systems that find text by meaning, not only by matching exact words. It covers turning documents into searchable number-based representations, splitting them into useful sections, choosing storage and indexes, and checking result quality.

In plain words
What is it for?
Use it to plan document-processing pipelines, choose chunking and embedding approaches, configure vector databases, combine keyword and meaning-based search, and evaluate retrieval results.
Why use it?
It helps avoid poor search caused by unsuitable document splits, language representations, indexes, or query assumptions. It also keeps the search design matched to document size, language, update rate, and expected questions.

Skill for Claude CodeCodex

Install

Getting it into your agent

One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.

agentmods
npx agentmods add skills/msdakot/ai-foundary/vector-database-engineer
Any agent
npx skills add msdakot/ai-foundary --skill vector-database-engineer
Clone the repo
git clone --depth 1 https://github.com/msdakot/ai-foundary

Made for: Claude Code, Codex.

Wrote this? Show the measurements

A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.

agentmods badge for vector-database-engineer

README.md
[![agentmods](https://agentmods.dev/badge/skills/msdakot/ai-foundary/vector-database-engineer.svg)](https://agentmods.dev/skills/msdakot/ai-foundary/vector-database-engineer)
Your own site
<a href="https://agentmods.dev/skills/msdakot/ai-foundary/vector-database-engineer"><img src="https://agentmods.dev/badge/skills/msdakot/ai-foundary/vector-database-engineer.svg" alt="Measured on agentmods" height="20"></a>
Per session 34 Skills are progressive disclosure: only the name and description are preloaded; the body loads when the skill is used.
When invoked 1,201 The whole file, excluding the scripts and references it only reads on demand.
Security scan A 0 findings. Scan, not verified.
Origin original No closer match found in the catalogue.
Token cost

What it costs to keep this loaded

Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.

ModelPer sessionOnce invoked
Fable 5 $0.00034 $0.01201
Opus 5 $0.00017 $0.00600
Sonnet 5 $0.00007 $0.00240
Haiku 4.5 $0.00003 $0.00120

Measured 3d ago against content hash b49de4692851, method: parsed. Prices are Anthropic first-party input rates as of 2026-08-30, from the pricing page.

Security

Grade A, and why

vector-database-engineer scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 3d ago.

A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.

Nothing flagged

None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.

agents/ai-data-agents/vector-database-engineer/SKILL.md · 129 lines

How it starts

The opening of the file, as written. The whole thing — 129 lines — stays where its author put it; the contents beside it link to each section on GitHub.

Vector Database Engineer Agent

You build semantic search and retrieval systems. You know that retrieval quality depends as much on chunking strategy and embedding choice as on index configuration.

Step 1 — Analyze the Corpus

Before designing the pipeline:

  • Document length distribution (short passages vs long documents)
  • Domain-specific terminology density (general vs specialized vocabulary)
  • Language distribution (mono vs multilingual)
  • Expected query patterns (short keyword-like vs natural language questions vs semantic similarity)
  • Scale: how many documents, update frequency, query volume

Step 2 — Chunking Strategy

Match chunking to content structure:

Content type Strategy
Unstructured text Fixed-size chunks (256–512 tokens) with 10–15% overlap
Structured documents (reports, papers) Semantic chunking at paragraph/section boundaries
Long documents requiring multi-resolution Hierarchical: summary chunk + detail chunks
Q&A or conversational Turn-level chunking
Code Function/class-level chunking

Never exceed the embedding model's effective context window — check the model card, not just max tokens.

Step 3 — Embedding Model Selection

Use case Model
General text similarity sentence-transformers/all-mpnet-base-v2
Speed-optimized sentence-transformers/all-MiniLM-L6-v2
Long documents (up to 8K tokens) nomic-embed-text, e5-mistral-7b
Code code-search-net, jinaai/jina-embeddings-v2-base-code
Multilingual paraphrase-multilingual-mpnet-base-v2
Image + text CLIP

Always evaluate candidate models on a representative benchmark from your corpus before committing.

Step 4 — Vector Store Selection

Need Store
In-process, high throughput FAISS
Managed cloud, production Pinecone, Qdrant
Hybrid vector + keyword Weaviate, Elasticsearch with dense vectors
Already running PostgreSQL pgvector
Self-hosted, metadata filtering Qdrant

Read the full file on GitHub · 129 lines

Changes

What this file has done since we first saw it

Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.

  1. 3d ago First seen · 129 lines · 34 tokens per session scan A b49de4692851

Subscribe to this mod's changes

vector-database-engineer is a skill published in the GitHub repository msdakot/ai-foundary (5 stars, last pushed 4mo ago), licensed MIT. It adds 34 tokens to every session and 1,201 once invoked, about $0.0002 per session on Opus 5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-31.

Related

Other skills, from other repositories

cli-anything-chromadb

Command-line interface for ChromaDB - A stateless CLI for managing vector database collections, documents, and semantic search. Designed for AI agents and automation via the ChromaDB HTTP API v2.

HKUDS/CLI-Anything · 47 tokens

qdrant-clients-sdk

Qdrant provides client SDKs for various programming languages, allowing easy integration with Qdrant deployments.

qdrant/skills · 28 tokens

neo4j-document-import-skill

Ingests unstructured and semi-structured documents into Neo4j as a knowledge graph. Use when chunking PDFs, HTML, plain text, or Markdown; extracting entities and relationships from text with an LLM (SimpleKGPipeline, neo4j-graphrag); loading JSON via apoc.load.json; building Document→Chunk→Entity graph structures; or…

neo4j-contrib/neo4j-skills · 187 tokens

neo4j-graphrag-skill

Build GraphRAG retrieval pipelines on Neo4j using the neo4j-graphrag Python package (v1.16.0+). Covers retriever selection (VectorRetriever, HybridRetriever, VectorCypherRetriever, HybridCypherRetriever, Text2CypherRetriever, ToolsRetriever), external vector DB retrievers (Weaviate, Pinecone, Qdrant), retrievalquery…

neo4j-contrib/neo4j-skills · 228 tokens

neo4j-vector-index-skill

Create and manage Neo4j vector indexes, run vector similarity search (ANN/kNN), store embeddings on nodes or relationships, use SEARCH clause (Neo4j 2026.01+, preferred) or db.index.vector.queryNodes() procedure (deprecated 2026.04, still works on 2025.x), configure HNSW and quantization options, pick similarity…

neo4j-contrib/neo4j-skills · 210 tokens

mongodb-search-and-ai

Guides MongoDB users through implementing and optimizing Atlas Search (full-text), Vector Search (semantic), and Hybrid Search solutions. Use this skill when users need to build search functionality for text-based queries (autocomplete, fuzzy matching, faceted search), semantic similarity (embeddings, RAG…

mongodb/agent-skills · 132 tokens