VLM-wiki: Instructions file for Codex

VLM Wiki Agent

Build and maintain a personal full-modal knowledge base using VLMs (Vision Language Models) and LLMs. Manage multiple directories for different media types, compile them into unified wiki articles. The wiki compounds over time.

Trigger keywords: vlmwiki, add media, add image, add video, add audio, multimodal wiki, 全模态, 记忆库

Model Note: If multi-modal models are not configured in .vlmwiki/config.json, the agent will automatically use the built-in models from the current AI IDE (e.g., Trae IDE's MiniMax-M2).

Core Philosophy

"The VLM wiki is a persistent, compounding artifact of your life." "The wiki writes and maintains itself with VLM/LLM; the human reads and reflects."

Architecture

Three layers, all under the user's project root:

project/
├── raw/                    # Immutable source material (read-only)
│   ├── images/            # Photos, screenshots, images
│   ├── videos/            # Video files with transcripts
│   ├── audio/             # Voice memos, recordings
│   ├── text/              # Text documents, notes
│   └── diary/             # Daily diary entries
│
├── wiki/                  # Compiled knowledge articles (VLM/LLM maintained)
│   ├── index.md          # Global index
│   ├── log.md            # Append-only operation log
│   ├── moments/           # Life moments & milestones
│   ├── people/            # People in your life
│   ├── places/            # Places you've been
│   ├── projects/          # Projects you're working on
│   ├── concepts/          # Topics, ideas, concepts
│   ├── patterns/          # Discovered patterns
│   └── archives/          # Archived syntheses
│
├── .vlmwiki/             # Configuration
│   └── config.json       # Model and settings config
│
└── AGENTS.md             # This file

Special Files

• wiki/index.md — Global index with one row per article, grouped by category
• wiki/log.md — Append-only operation log
• .vlmwiki/config.json — Model configuration (VLM, LLM, storage paths)