grounding-ai: Instructions file for Claude Code

CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

CRITICAL REQUIREMENTS

Python Version

REQUIRED: Python 3.13.x - Do NOT use Python 3.14+

Python 3.14 is blocked by unstructured (requires-python: <3.14). Always verify:

python3.13 --version  # Must show 3.13.x
./venv/bin/python --version  # Verify venv uses 3.13

Embedding Generation is MANDATORY

IMPORTANT: Any script or process that ingests documents into the corpus MUST include embedding generation as a final step. Documents without embeddings are not searchable by agents.

When writing ingestion scripts:

1. Process documents into corpus
2. Identify affected collections
3. Update embeddings for all agents that use those collections

Example pattern:

# After ingestion, update embeddings for affected agents
for agent in mathematician data-scientist scientist; do
  ./venv/bin/grounding embeddings --agent "$agent" \
    --corpus /path/to/corpus \
    --agents-dir ./agents \
    --out /path/to/embeddings/"$agent" \
    --incremental
done

The staging watcher handles this automatically when AUTO_EMBEDDINGS=true, but manual/batch ingestion scripts must explicitly include this step.

---

Project Overview

Grounding (PyPI: grounding-ai) is a local-first document corpus pipeline for grounded AI agents. It converts documents into LLM-ready Markdown artifacts with structured metadata, deterministic chunking, and corpus-level manifest generation, using open-source libraries (Unstructured, Marker, LangChain).

Supported formats: PDF, EPUB, Markdown (.md), Word (.docx, .doc)

Key Design Principles

• Determinism: Same inputs produce byte-identical outputs (excluding timestamps)
• Error resilience: Per-file error handling; batch processing continues on failures
• Local-only: No network calls; privacy-focused
• Provenance: Content hashing and tool version tracking in metadata

Virtual Environment