Spelunking - Codebase Exploration

Spelunking - Persistent Codebase Exploration

Spelunking is a system for deep, targeted codebase exploration that produces persistent, reusable documentation. Instead of each agent re-reading the same files, spelunk docs are generated once and reused until the underlying code changes.

Core Concept

Agent needs to understand code area
  → Check if spelunk doc exists for that area
    → If YES and FRESH: read the doc, skip exploration
    → If YES and STALE: re-explore, update the doc
    → If NO: explore, create the doc

Document Storage

docs/spelunk/
  ├── contracts/          # Type definitions, API signatures (for Architect, QA)
  ├── flows/              # User flows, entry points (for Product)
  ├── boundaries/         # Module edges, dependencies (for Architect)
  ├── trust-zones/        # Auth boundaries, data flow (for Security)
  ├── _index.md           # Human-readable navigation index
  └── _staleness.json     # Machine-readable staleness tracking

Hash-Based Staleness Tracking

Each spelunk document tracks the source files it was generated from:

// docs/spelunk/_staleness.json
{
  "docs/spelunk/flows/user-authentication.md": {
    "generated": "2025-01-10T14:30:00Z",
    "source_files": {
      "src/auth/login.ts": "a1b2c3d4",
      "src/auth/middleware.ts": "e5f6g7h8",
      "src/routes/auth.ts": "i9j0k1l2"
    }
  }
}

Staleness Check Algorithm

1. Read _staleness.json entry for the target doc
2. For each source file:
   a. Compute current hash: first 8 chars of SHA-256
   b. Compare with stored hash
3. If ANY hash differs → STALE
4. If all hashes match → FRESH
5. If no entry exists → MISSING (needs full exploration)

Computing File Hash

# First 8 characters of SHA-256
sha256sum src/auth/login.ts | cut -c1-8

Spelunk Command Syntax

spelunk --for=<agent-type> --focus="<area description>"

Options:
  --for       Target agent lens: architect, product, qa, security
  --focus     Natural language description of exploration area
  --check     Only check staleness, don't re-explore
  --force     Re-explore even if FRESH