Recursive Language Models

An RLM is a system around a language model for reasoning over prompts that are too large or too complicated to handle reliably in one pass. The long prompt stays outside the model as an environment it can inspect.

Inference-time systems

Suppose a university gives you a year of maintenance reports and asks how many rooms had an active water leak after a repair. One report says a ceiling tile was still stained. Another says a drip started again two days after the plumber left. Neither report uses the word “leak,” but one describes an active leak. And if two reports describe the same room, you still have to count it only once.

You could give the entire archive to a language model and ask for a number. A recursive language model (RLM) approaches the same question by letting the model write code to inspect the archive, ask other model calls to interpret difficult passages, and count the resulting room IDs. The archive stays available while the model works, so it can go back for evidence when an answer looks wrong.

What goes into the model?

In a regular model call, the question and the reports go into the context window together. The RLM in Zhang, Kraska, and Khattab’s paper keeps the source in a Python environment. The root model gets a way to access the source and can choose what to inspect. Each individual model call still has its own context limit.

The Python environment is a read-eval-print loop, or REPL. Think of it like a notebook where variables survive between cells. The root can write one cell to separate the reports by room, then write a second cell that uses those groups. The root sees what the cells print, which the runtime can truncate. The full groups can stay in Python without being copied into the root model’s context.

A model call is also available as a Python function. That means a single cell can loop over many rooms, ask a model a question about each one, and save the results. The root can then look at a handful of uncertain cases before it decides what to do next.

A search tool would be enough to find a particular room’s report if you already knew its ID. Counting every room that meets a condition asks for wider coverage. A few good search results may leave dozens of rooms unchecked, and a summary of the archive might leave out dates or repeated visits. The REPL lets the model choose between a targeted search and going through every room in a loop.

Try it on three rooms

Here is a tiny version of the maintenance archive. Each line stands for a dated entry. Assume the question is whether any follow-up visit documented an active leak after a repair in the same room.

A12, May 03: Replaced supply-line fitting.
A12, May 05: Water dripped from the new fitting.
A12, May 06: Drip from the fitting still present.

B07, May 03: Patched the roof above this room.
B07, May 05: Old ceiling stain remains. Surface dry.

C02, May 03: Puddle under the sink. Repair scheduled.
C02, May 05: Replaced sink trap. No further report.

Group the entries by room and keep them in date order. Each helper then sees the repair and its follow-up together.
Ask the helper to return yes, no, or unknown, along with the entry that supports its label. A12 is yes. B07 is no based on the dry follow-up. C02 is unknown because the record ends at the repair.
Count the room IDs labeled yes. The answer is one. Keep C02 in a separate list so the final answer can say that one room lacked a follow-up.

Where the model helps and where code helps

Grouping and counting are jobs for Python once the records have a consistent format. The language model does the harder reading. “Stain remains” and “water dripped” describe different situations, and a helper can make that distinction in the context of a room’s full record.

The sketch below assumes the records are already grouped and the helper returns a parsed label and evidence. In a real run, the root would decide how to group the records after looking at the input. It could also ask a second question about any room that came back as unknown.

# Illustrative Python. classify_room wraps a model call.
decisions = {}
for room, entries in entries_by_room.items():
    decisions[room] = classify_room(question, entries)

confirmed = [
    room for room, result in decisions.items()
    if result["label"] == "yes"
]
print(len(confirmed))  # 1 for the sample archive

What makes the calls recursive?

In the example, the root calls a helper model for each room. That is one level of delegation. If a room has an unusually large file, the helper could itself be an RLM with its own workspace. It might split the file by repair period and ask a model about each period before returning a room-level answer. The process stops when a call can answer directly or hits its budget.

Ten helper calls from the root are still one level deep. A helper that launches its own calls makes the tree two levels deep. This distinction matters when comparing runs, because the depth limit says very little about the number of calls or their total cost.

What the paper found

The OOLONG benchmark asks models to infer labels from rows of text and then combine those labels to answer a question. That is closer to the maintenance example than a search for a single fact. In Table 1 of the paper, direct GPT-5 scored 44.0 on OOLONG. The depth-one RLM with GPT-5-mini helpers scored 56.0. The Python-only variant scored 36.0 on the same task.

The comparison changes on CodeQA. With Qwen3-Coder, the variant that could inspect the input in Python without calling helper models scored 66.0, ahead of all three tested RLM variants that allowed sub-calls. Whether delegation helps depends on what work the task actually needs.

Where the approach can go wrong

A correct count can still rest on wrong labels. If the root groups A12’s May 3 repair and May 5 follow-up into separate batches with no shared ID, a helper can miss the fact that the drip came back. If the root drops all entries that never say “leak,” the missing evidence is unavailable to every helper it calls. Keeping room IDs and source entries with each result lets the root check where a label came from.

The cost is also uneven. A simple question might take one search and two model calls. An ambiguous archive can lead to repeated reads and a long string of calls. The paper found cheaper median runs than direct model calls in its cost analysis, but expensive runs raised the average. A direct model call makes a useful baseline when the document fits, and a Python-only version helps show what the helper calls add.