An AI research agent has already tried hundreds of ways to solve a problem. Some worked, some failed, and every attempt cost something. Before paying for another round, could the system learn to choose its attempts better?
That is the useful question behind Dream-RSI, a preprint posted on September 14. Its version of self-improvement changes the software directing a search, not the foundation model doing the work. The distinction matters: better management of a fixed model is a meaningful result without being evidence of an intelligence explosion.
The part that learns is the search controller
The authors separate a discovery agent, which proposes candidate solutions, from an exploration policy: executable code that decides where the agent should look next. That policy can change how a search branches, which promising candidates receive more work, how parallel attempts are allocated and when exploration stops. The model and evaluator remain fixed.
After an online discovery round, the system stores the resulting search tree. An offline stage replays decisions against those recorded outcomes while a policy-development agent revises the controller. The next online round uses the revised policy and adds new experience. The project’s technical explanation describes this alternating process as expanding both the search history and the policy that navigates it.
Consider two possible improvements to an automated coding workflow. One makes its language model better at writing code. The other notices that the workflow repeatedly spends time refining weak candidates and changes when it abandons them. Both could improve the final result. Only the first necessarily changes the model’s underlying capabilities. Dream-RSI investigates the second kind of leverage.
A narrower, more useful reading of the numbers
In an experiment finding faster solvers for Lasso, a statistical method that selects useful variables, Dream-RSI used 317 discovery-agent calls with Gemini 3.1 Pro, compared with 550 for fixed exploration. The resulting solver also had a lower average runtime on six held-out datasets: 2,931 rather than 3,587 milliseconds. These are author-reported preprint results, not an independent replication.
There are two different clocks here. Agent calls measure part of the effort spent finding a program. Runtime measures how quickly the discovered program later executes. Neither number alone tells a buyer what the entire discovery process costs. Policy development, replay infrastructure and evaluation must also fit inside that accounting boundary.
The authors’ repository lists eight experimental tasks across algorithm discovery, mathematical optimization and GPU kernels—small routines executed on graphics processors. At the time of checking, it said code, discovered programs and reproduction scripts were still being prepared for release. That leaves an important step between a readable research claim and a result outside teams can readily reproduce.
The replay cannot see the road not taken
A stored search tree is not a complete model of the world. Replay can reuse outcomes on recorded branches; it cannot reveal the result of an unrecorded experiment. A controller that looks better on yesterday’s tree could still make poor choices on tomorrow’s task. Historical replay performance is not a guarantee of future discovery.
This gives the method a practical tension. Reuse enough history and the system may avoid waste. Depend too heavily on that history and it may favor familiar routes. Our interpretation is that the valuable test is not whether the controller becomes more elaborate, but whether its improvements survive genuinely new tasks under complete cost accounting.
Vastkind’s earlier look at recursive self-improvement asks who controls the wider research loop. This paper adds a bounded mechanism to that discussion. It also differs from Retrieve-for-Train, which moves retrieval planning into training, and from materials-discovery systems, where a shortlist still has to face laboratory evidence.
The next persuasive result would show that replay-guided search saves resources on unfamiliar problems after every part of the improvement loop has been counted.
