An AI Fixed Another Model’s Reasoning Loops by Editing Its Weights

A small AI model kept getting stuck while working through math problems. It repeated itself until it ran out of room to answer. Researchers gave a stronger AI access to its internal numbers and asked it to make repairs. On one math benchmark, the edited model’s score rose from 7.9% to 48.1%.

The striking part is how the improvement happened. The stronger model did not run a conventional training process. It inspected and altered selected weights, the numerical settings that shape how a neural network processes information. The researchers’ evidence points to a repair of repetitive reasoning, not a general method for teaching an AI new mathematics.

That distinction is the most useful finding in Weight-Space Programming, a research report published November 2 by Rikiya Takehi, Jayden Teoh, Alex Zhang, and Omar Khattab at MIT CSAIL. Its follow-up experiment also shows where the approach failed.

Editing the Model, Not Its Instructions

A prompt changes what a model is asked to do. Editing weights changes the model itself: the learned numerical relationships it uses to produce a response. Conventional fine-tuning adjusts those numbers through repeated examples and an optimizer that calculates how to reduce errors.

Here, the researchers prohibited that training process. GPT-6 Astra instead worked through a Python environment with access to the weights of Qwen3.5-0.8B, a model with roughly 800 million parameters. It could inspect the numbers, propose edits, and test behavior on practice problems whose answers were available. It could not use the final benchmark questions during that work.

In the strongest reported run, Astra identified a recurring pattern in two internal layers associated with the model’s loops. It then reduced that pattern in a small selection of weights. The edit touched 131,072 entries, about 0.016% of the model’s stated size. After conversion to the numerical format used for evaluation, 130,188 entries actually differed from the original.

This was not an AI improving itself. A larger model edited a different, smaller model inside a research setup with defined tools, restrictions, and evaluation.

A Better Score Can Begin With Reaching an Answer

The clearest comparison is within Qwen, before and after editing under the same evaluation settings. On MATH-500, a set of 500 math problems, accuracy increased by 40.2 percentage points. The scores averaged four generated answers per problem, rather than taking only the best answer.

At the same time, the share of MATH-500 answers that reached the 32,768-token limit fell from 93.0% to 33.1%. Tokens are the chunks of text a model reads or writes; hitting the limit can cut off a response before it gives an answer. A cutoff is not itself proof of a loop, but the researchers also inspected repetitive outputs and traced the behavior they were trying to remove.

The repair helped on other tests too. GSM8K math accuracy rose from 13.8% to 67.1%, and the reported instruction-following score on IFEval rose from 21.0% to 62.3%. The authors interpret this as removing an obstacle that affected more than mathematics.

The results are promising because useful reasoning can be wasted by a failure to finish. They do not establish that every extra correct answer reflects a newly acquired skill. Nor did the repair eliminate the problem: roughly a third of MATH-500 responses still reached the limit.

The Model That Did Not Have the Same Problem

The researchers then tried editing Gemma 3 270M, a smaller model that much less often reached its answer-length limit. Its mistakes looked more like ordinary arithmetic and algebra errors than a single repetitive behavior that could be switched off.

The large improvement did not repeat. Using the report’s grading method for Gemma, MATH-500 accuracy went from 6.8% before editing to 7.0% in the 50-test-pass run. Other tested budgets reached at most 7.7%. The researchers did not establish a meaningful improvement.

Those percentages are not a fair ranking of Gemma against Qwen. The models differed in size, sampling, response limits, and answer grading. Gemma’s grader could recognize answers outside the requested final-answer format; the Qwen evaluation used the last boxed answer. The useful comparison is each model against its own baseline.

Even with that boundary, the contrast matters. Finding a bottleneck that suppresses an existing ability is a different task from creating an ability that is missing. The successful loop repair demonstrates the first more clearly than the second.

A Small Edit Is Not Necessarily a Cheap Repair

“No training” does not mean “no testing.” The strongest Qwen run used a budget of 50 test passes, and each pass could contain a batch of hundreds of practice problems. The released logs record 80 Python runs and 158 conversational turns. Its 20.3-hour wall-clock duration included a 4.7-hour interruption when the editing model could not be reached.

That is not a clean measure of compute cost, but it rules out reading the result as an instant one-shot fix. The work also does not prove that a small edit leaves every untested capability intact.

The authors released experiment logs, evaluation summaries, and reproducible weight edits. The results discussed here are theirs, not an independent Vastkind rerun. For a related measurement problem, our explanation of changing AI benchmarks shows why the test matters; our look at AI research judges examines how evaluation instructions can change a verdict.

Weight editing offers a different way to work on learned software: locate a specific failure inside the numerical machinery and repair it directly. In this experiment, the breakthrough was less like teaching a student new mathematics than helping one stop repeating the same line long enough to finish the answer.