compact-memory-implementation

compact-memory-implementation

A developer guide for building compact memory into an Agent: detect when to compress, fork a compactor sub-agent, produce a structured summary, and restore it in the next session.

Step 1 — Understand the setup

Before designing anything, clarify:

• SDK / language: Claude Agent SDK? Direct Anthropic API? Python or TypeScript?
• Agent architecture: single-agent loop, multi-agent, tool-calling?
• Session model: one long-running session or multiple short sessions?
• What must survive compaction: task state, decisions, tool results, conversation history?

This determines which pattern fits.

Step 2 — When to trigger compact

Three strategies, pick based on your session model:

1. Token threshold (recommended) Check usage.input_tokens from the previous response. When it exceeds ~70–80% of your model's context limit, trigger compact.

COMPACT_THRESHOLD = 150_000  # adjust per model

if response.usage.input_tokens > COMPACT_THRESHOLD:
    compact = compact_memory(history)
    history = []  # reset — compact moves to system prompt

2. Turn count Compact every N turns. Simpler but less adaptive — misses sessions with a few very long turns.

COMPACT_EVERY_N = 30

if turn_count % COMPACT_EVERY_N == 0:
    compact = compact_memory(history)

3. Phase boundary Compact at natural task boundaries (after research, before implementation). Requires the agent to detect phases. Produces summaries that align with meaningful milestones, but harder to implement reliably.

Recommended default: token threshold at 70%, with turn-count fallback at N=40.

Step 3 — Fork agent for compaction

The compactor is a separate agent call whose only job is to read the current state and return a structured summary. Fork it synchronously — the main agent waits for the result before continuing.

def compact_memory(history: list[dict]) -> dict:
    response = client.messages.create(
        model="claude-haiku-4-5-20251001",  # cheaper model is fine for compaction
        max_tokens=4096,
        system=COMPACTOR_SYSTEM_PROMPT,
        messages=[
            {
                "role": "user",
                "content": format_history_for_compact(history),
            }
        ],
    )
    return json.loads(response.content[0].text)