memsearch CLAUDE.md

CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

Build & Test Commands

# Install in development mode
uv sync --all-extras

# Run all tests (use python -m pytest to avoid system pytest conflicts)
uv run python -m pytest

# Run a single test file
uv run python -m pytest tests/test_chunker.py

# Run a specific test
uv run python -m pytest tests/test_store.py::test_upsert_and_search -v

# Serve docs locally
uv run mkdocs serve

# Run the CLI
uv run memsearch --help

Architecture

memsearch is a semantic memory search engine for markdown knowledge bases, built on Milvus.

Data Flow

Markdown files → Scanner → Chunker → Embedder → MilvusStore
                                                      ↓
                               User query → Embedder → Hybrid Search (dense + BM25 + RRF) → Results

Core Library (src/memsearch/)

• core.py — MemSearch class: the public Python API that orchestrates everything. Entry point for index(), search(), compact(), watch().
• store.py — MilvusStore: Milvus wrapper handling collection creation, upsert, hybrid search (dense cosine + BM25 sparse + RRF reranking), and cleanup. The chunk_hash (composite ID of source+lines+content+model) is the VARCHAR primary key.
• chunker.py — Splits markdown by headings into Chunk dataclasses. SHA-256 content hash enables dedup. compute_chunk_id() generates composite IDs matching OpenClaw's format.
• embeddings/__init__.py — EmbeddingProvider protocol + lazy-loading factory (get_provider()). Providers: openai (default), google, voyage, jina, mistral, ollama, local, onnx.
• scanner.py — Walks directories to find .md/.markdown files, returns ScannedFile list.
• config.py — Layered TOML config: dataclass defaults → ~/.memsearch/config.toml → .memsearch.toml → CLI flags.
• cli.py — Click CLI wrapping the Python API. All commands resolve config via resolve_config() then instantiate MemSearch.
• watcher.py — watchdog-based file watcher with debounce, used by memsearch watch and the Claude Code plugin.
• compact.py — LLM-powered chunk summarization (OpenAI/Anthropic/Gemini).
• reranker.py — Optional cross-encoder reranking (ONNX or PyTorch backend). Disabled by default; enable via reranker.model config.