recursive-language-model

Recursive language model (decomposition harness)

A methodology, not a library — the coding agent writes the loop to fit the workload. Use it when a frontier model does a task in one giant agentic prompt (many tool declarations + a context that compounds as raw tool results pile up) that a smaller local model cannot. Instead of shrinking the model, change the harness: have the small model solve the task as a sequence of small, bounded steps, each with a deliberately flat context.

Crucially, the RLM wraps the small model behind the incumbent's existing call contract — the workload's callers send the same request and get the same answer; the decomposition happens inside. That makes the specialist a drop-in replacement for the frontier generalist, not a rewrite of the system around it.

This composes two other skills: it needs ../understand-workload/SKILL.md to know the task, its steps, and the success criteria, and ../design-simulated-environment/SKILL.md so the small model has a real, scorable place to run the whole case.

The two things that make the giant prompt fail for a small model

• Fixed tool overhead. The whole tool catalog is re-sent every turn, often the large majority of the prompt, for a turn that uses one tool. Fix: per-step tool subsetting — show the model only the one or few tools that step needs.
• Compounding context. Each raw tool result (some very large) is appended and re-sent on every later turn, so the same bytes are billed many times and the window fills. Fix: summarize each result into the few facts the task needs and carry only the summary forward — context stays small and roughly flat.

Recipe

1. Frame the loop. State = the task + a short scratchpad of notes + the step number. Each step, give the model only: the task, the relevant tool subset, and the current scratchpad. Ask for exactly one next action (or "done").
2. Subset the tools per step from the workload's tool catalog — never the full surface. Pick by intent (the step's verb → its tool class).
3. Run the action against the simulated environment (so there are no live side effects and any tool call gets a result), then summarize the result into the scratchpad. The raw result must not re-enter the next prompt — summarizing well is where recall is won or lost; extract exactly the fields the gold needs.
4. Recurse when a step is itself big. A step may be "spawn a sub-loop on sub-task X" (mirroring the workload's own sub-agents) — a fresh small-context loop whose summary returns to the parent. This is the "recursive" in RLM.
5. Stop when the model writes its answer/observations or hits a step budget; score the final state with the environment's validator.