chimera-budget-the-context

Trigger

You are running a loop that appends to a message list every step — a ReAct agent, a coding turn, a tool-calling assistant — and the list only ever grows. Symptoms: the provider raises on context length, or the run survives but starts working on the wrong file after a long stretch.

It does not apply to a single-shot call whose prompt you assembled yourself and can measure once. There is no eviction policy to design when nothing accumulates.

Do

1. Write down two numbers before touching the loop: the model's advertised window, and the fraction of it you will spend on the prompt. Chimera spends 0.6 (DEFAULT_BUDGET_FRACTION in chimera/core/context_budget.py). The rest pays for the completion and for the gap between your token estimate and the provider's real count.
2. Fire compaction on a share of the budget, not of the window — Chimera triggers at 0.8 of the budget. Compaction needs room to compact into; a trigger set at the window fires when there is none left.
3. Measure the fixed floor separately from the growing part. The system message plus the tools= payload are re-sent on every step and no amount of message compaction touches them. Run chimera schema-bench (or count the JSON yourself) before you assume the history is what is expensive. A verbose MCP or OpenAPI import can put tens of thousands of tokens under every single step of the run.
4. If the floor is a problem, shrink the floor: strip annotation-only schema keys (examples, title, default, $comment) and trim parameter prose to the first sentence, keeping type, properties, required and enum intact so tool selection and argument validity are unchanged. That is what chimera/tools/schema_compact.py does. Advertise fewer tools if it is still too big.
5. Fix the eviction order explicitly: system message never, recent turns verbatim (Chimera keeps 6), the older span replaced by a summary or a factual note. Never rewrite the system message — it is the stable prefix the prompt cache is keyed on, and editing it invalidates every cached turn behind it.
6. After compacting, re-inject what the run needs to still be itself: the task verbatim, the plan, the task list with status, and the file currently being edited. Re-read the file from disk rather than restoring a remembered copy.
7. When you estimate token size yourself, count the tool_calls payload as well as content. Tool arguments ride on the assistant message and are not in content; a size estimate that reads only content under-reports exactly the messages that grew.