astra-knowledge-base-mcp: Instructions file for Codex

astra-knowledge-base-mcp — Agent Guide

For AI agents developing and extending this project. Humans can skip to README.

Project Overview

MCP (Model Context Protocol) server for managing multi-tenant knowledge bases. Part of Astra AI Agent Infrastructure.

Key architectural choices:

• PostgreSQL 16+ with pgvector is the single, authoritative backend (keeps dev and prod on identical storage semantics).
• Embedding is provider-agnostic: config via ASTRA_EMBED_BASE_URL + ASTRA_EMBED_API_KEY + ASTRA_EMBED_MODEL. Any OpenAI-compatible /v1/embeddings endpoint works — local llama.cpp, SiliconFlow, OpenAI, DeepSeek, etc.
• SAG (SQL-Retrieval Augmented Generation, arxiv 2606.15971, MIT) is the retrieval architecture we are adopting — event-entity indexing + query-time dynamic hyperedges via SQL JOINs.
• Search strategies are additive — new paths (sag_fast, sag_precise) coexist with existing ones (fts, vector, hybrid), exposed through a unified kb_search interface.
• Self-implemented, not wrapping zleap-sag — we implement the SAG algorithm directly on our PG schema. The zleap-sag package is a dev dependency for reference/verification only.

Code Map

astra-knowledge-base-mcp/
├── server.py                 # MCP server entry — tool definitions & dispatch
├── pg_backend.py             # PostgreSQL backend — KB lifecycle, chunks, search (THE backend)
├── embed_client.py           # Embedding client — provider-agnostic, OpenAI-compatible
├── chunking/
│   ├── __init__.py
│   ├── base.py               # Chunker ABC
│   └── recursive.py          # RecursiveChunker — paragraph/sentence splitting
├── ingestion/
│   ├── __init__.py
│   ├── base.py               # Ingestor ABC
│   └── text.py               # TextIngestor — text/file → chunks
├── search/
│   ├── __init__.py
│   ├── engine.py             # SearchEngine ABC (pluggable interface)
│   └── fts.py                # FTS search implementation
├── sag/                      # [Phase 1] SAG retrieval module (to be created)
│   ├── __init__.py
│   ├── extractor.py          # LLM-based event/entity extraction
│   └── search.py             # SAG retrieval pipeline
├── docs/
│   └── kb-wiki-interop.md    # Two-layer interop reference doc
├── scripts/
│   ├── run.sh                # Startup script
│   ├── wiki-kb-sync.sh       # One-click wiki → KB sync script
│   ├── dir-kb-sync.py        # [Phase 3] Whole-directory vectorisation CLI (generalised from wiki-kb-sync)
│   └── classify-input.sh     # Input classification — detect wiki vs doc stack vs single file, handle archives
├── templates/
│   └── kb-wiki-page.md       # KB-optimised wiki page template
├── plugins/
│   └── memory/
│       └── astra_kb/         # [Phase 2] Hermes memory provider (ships standalone)
├── skills/                   # Skills for AI agents (symlinked from ~/)
│   └── knowledge-base-interop/
│       └── SKILL.md          # Two-layer interop skill
├── AGENTS.md                 # This file (tracked, sanitised) — includes evolution roadmap below
├── PLAN.md                   # Long-term development roadmap (read before starting work)
├── README.md
├── pyproject.toml
└── .venv/                    # Virtual environment (uv-managed)