GPT-6 Astra edited a small Qwen model's weights and cut its runaway loops

What happened

Rikiya Takehi, Jayden Teoh, Alex Zhang and Omar Khattab of MIT CSAIL published "Sparks of Weight-Space Programming" on November 2. It's a blog post with logs, not a peer-reviewed paper. In eight experiments they hand OpenAI's GPT-6 Astra the weights of a neural network and ask it to say what the network does, fix it or write a new one, without letting it run a training loop. The last one, making a small language model better at math, drew 2,900 points on Hacker News.

The Qwen experiment

Astra got a Python session with Qwen3.5-0.8B loaded, no GPU, practice problems from the GSM8K and MATH training sets, and a fixed budget of 10, 25 or 50 forward passes to try its edits on. No gradients. The test benchmarks stayed hidden from it.

It found that the model loops: 85 of its first 96 practice answers ran to the 32,768-token limit. It traced the loops to one direction written by the MLPs of layers 15 and 19 and subtracted it from 64 output columns in each, which is 131,072 of the model's 0.8 billion weights.

Read the last column. The jump at 50 passes comes with the share of runaway answers falling from 93% to 33%. The authors say so: fixing loops lets the model finish answers it could already work out. Run on Gemma 3 270M, which rarely loops, the same setup moved MATH-500 by about a point at most.

The edited Qwen is on Hugging Face.

The pushback

Within a day Matthew Honnibal, who created spaCy, posted a reply calling the framing misleading. He argues that only three experiments have Astra read raw weights, all on networks of at most 172K weights, and that the failed runs get less space than the wins. He still calls the Qwen fix real.

What it means for you

1. You run small open models locally: if one keeps rambling to the token limit, that's what the authors call a common failure of tiny reasoning models, and they note that Qwen's model card warns about it for this one. Check the model card's sampling settings first; the authors held theirs fixed (temperature 0.6, top-p 0.95, top-k 20, presence penalty 1.5), so the gain in the table came from the edit alone.
2. You read AI benchmark headlines: look for a column like "hit token limit". A score that jumps because a model stopped looping hasn't made it smarter.

Try it

I want you to act as a skeptical ML reviewer. Below is a claim about a model's benchmark gain and the table or text behind it.
1) Restate the claim in one sentence with its numbers.
2) List what else changed between the two runs: sampling settings, token limits, prompts, number of samples, test set.
3) Name the simplest explanation for the gain that isn't "the model got more capable", and the number in the source that would confirm or rule it out.
4) Say what the source itself admits as a limit.
Quote the source for every number. If something isn't in the text, say "not stated".

[paste the claim and the source text]

I want you to act as a local-LLM troubleshooter. My model keeps repeating itself or runs to the token limit without answering.
Model and size: [model]. Runtime: [Ollama, llama.cpp, LM Studio, vLLM]. Current settings: [temperature, top-p, top-k, repetition or presence penalty, max tokens]. Thinking mode: [on or off].
Here's one looping output: [paste the end of it].
Tell me which setting to change first and to what, based on the model card if you know it, then what to try if that doesn't work. One change at a time, and say how I'll know it helped.

Sources

• Takehi, Teoh, Zhang and Khattab (MIT CSAIL), Sparks of Weight-Space Programming, November 2, 2026
• Matthew Honnibal, The 'Astra can read and write weights' post is misleading, checked November 3, 2026
• Hugging Face, RikiyaT/Qwen3.5-0.8B-weight-space-programming, checked November 3, 2026