iterative-retrieval

Iterative Retrieval Pattern

Solves the "context problem" in multi-agent workflows where subagents don't know what context they need until they start working.

The Problem

Subagents are spawned with limited context. They don't know:

• Which files contain relevant code
• What patterns exist in the codebase
• What terminology the project uses

Standard approaches fail:

• Send everything: Exceeds context limits
• Send nothing: Agent lacks critical information
• Guess what's needed: Often wrong

The Solution: Iterative Retrieval

A 4-phase loop that progressively refines context:

┌─────────────────────────────────────────────┐
│                                             │
│   ┌──────────┐      ┌──────────┐            │
│   │ DISPATCH │─────▶│ EVALUATE │            │
│   └──────────┘      └──────────┘            │
│        ▲                  │                 │
│        │                  ▼                 │
│   ┌──────────┐      ┌──────────┐            │
│   │   LOOP   │◀─────│  REFINE  │            │
│   └──────────┘      └──────────┘            │
│                                             │
│        Max 3 cycles, then proceed           │
└─────────────────────────────────────────────┘

Phase 1: DISPATCH

Initial broad query to gather candidate files:

// Start with high-level intent
const initialQuery = {
  patterns: ['src/**/*.ts', 'lib/**/*.ts'],
  keywords: ['authentication', 'user', 'session'],
  excludes: ['*.test.ts', '*.spec.ts']
};

// Dispatch to retrieval agent
const candidates = await retrieveFiles(initialQuery);

Phase 2: EVALUATE

Assess retrieved content for relevance:

function evaluateRelevance(files, task) {
  return files.map(file => ({
    path: file.path,
    relevance: scoreRelevance(file.content, task),
    reason: explainRelevance(file.content, task),
    missingContext: identifyGaps(file.content, task)
  }));
}

Scoring criteria:

• High (0.8-1.0): Directly implements target functionality
• Medium (0.5-0.7): Contains related patterns or types
• Low (0.2-0.4): Tangentially related
• None (0-0.2): Not relevant, exclude