vector-databases

Vector Databases

Vector databases are the retrieval layer for semantic search and RAG (Retrieval-Augmented Generation) systems, storing high-dimensional embedding vectors and enabling approximate nearest-neighbor (ANN) search at scale. This skill covers the full engineering stack: choosing and configuring a vector store, building embedding pipelines, tuning ANN indices for the accuracy/latency/memory trade-off, implementing hybrid retrieval, and wiring everything into production RAG architectures. It orbits the llm-integration and knowledge skills.

Core Concepts

Embedding Pipelines

Embeddings are the foundation — garbage in, garbage out. Model selection matters: text-embedding-3-large (OpenAI, 3072 dims, reducible via dimensions param), voyage-3 (Voyage AI, strong for code/technical), embed-english-v3.0 (Cohere, supports int8 quantization natively), nomic-embed-text-v1.5 (open-source, Matryoshka — supports variable output dims). For multilingual: multilingual-e5-large or Cohere's multilingual model.

Batching: always batch embedding calls. OpenAI allows up to 2048 inputs per call. Local models via sentence-transformers: model.encode(texts, batch_size=64, show_progress_bar=True, normalize_embeddings=True). Normalize to unit sphere if using cosine similarity (makes dot product equivalent, faster in FAISS/Qdrant).

Chunking strategy is retrieval-critical: fixed-size (512 tokens with 50-token overlap) works as a baseline. Semantic chunking (split on embedding similarity drops) improves coherence. For structured docs (PDFs, HTML), prefer element-aware chunking via unstructured library — respects headings, tables, lists. Store chunk_index and parent_doc_id as metadata to enable parent-document retrieval (fetch surrounding chunks at query time).

FAISS

FAISS is the reference ANN library (Meta). Index types:

• IndexFlatL2 / IndexFlatIP: exact brute-force, no approximation. Use for < 1M vectors or as ground truth for recall benchmarking.
• IndexIVFFlat: inverted file index. nlist = number of Voronoi cells (rule of thumb: sqrt(N) to 4*sqrt(N)). At query time, nprobe cells searched — trade recall for speed. Requires training: index.train(vectors).
• IndexIVFPQ: adds Product Quantization compression. m subquantizers (must divide d), nbits=8 standard. Dramatic memory reduction (32x with m=d/4, nbits=8) at modest recall cost.
• IndexHNSWFlat: HNSW graph index. M = edges per node (16-64, higher = better recall + more memory), efConstruction = search width during build (200-500). At query time: index.hnsw.efSearch (32-256). No training required.
• IndexIVFPQ + HNSW coarse quantizer: best recall/memory/latency for large-scale production.