arcane: Instructions file for Codex

Arcane — Agent Guidance

This file tells AI coding agents (Claude Code, Cursor, Copilot Workspace, etc.) how to work with this codebase effectively.

Project Overview

Arcane is a personal engineering memory system exposed as an MCP (Model Context Protocol) server. It stores decisions, bugs, patterns, and learnings across engineering sessions using SQLite with FTS5 + vector search.

Architecture

src/arcane/
├── cli/            # Click CLI — one module per command group
├── domain/         # Pydantic domain models (Memory, Journey, Artifact…)
├── infra/          # Infrastructure: DB repos, config, search, embeddings, redaction
│   ├── db/         # SQLite schema, MemoryRepo, JourneyRepo, ArtifactRepo, RelationshipRepo
│   └── embeddings/ # Ollama + OpenAI embedding providers
├── mcp_server/     # MCP stdio server + tool handlers
│   └── tools/      # One file per tool group (memory, journey, artifact…)
├── plugins/        # Plugin system (ingestion, intelligence, content)
│   └── builtin/    # Built-in plugins: git, GHA, Linear, velocity, blog, ADR
└── services/       # Business logic: MemoryService, JourneyService, ArtifactService…

Key Conventions

Dependency Injection

All services receive a ServiceContainer (defined in services/container.py). Never construct repos or the DB directly outside of create_container() or tests. CLI commands must use the with create_container() as container: pattern for proper resource cleanup.

Repository Boundary

All DB read methods return fully deserialized Python objects — tags are list[str], datetimes are strings. Never call json.loads() on data returned from a repo method; _process_row() in MemoryRepo handles this centrally.

Embedding + Vector Search

Vector search is optional — the system degrades gracefully to FTS if memories_vec table is absent or dimensions mismatch. The dimension is stored in the meta table and checked on every embed operation. Use memory_repo.invalidate_vec_cache() after DDL changes to the vec table. Never ALTER TABLE ... RENAME a vec0 table: its shadow tables keep the old name and the renamed table stops working — drop and recreate it inside one transaction instead.