MIT researchers show GPT-6 Astra can read and edit neural network weights directly

A team at MIT CSAIL has published early results suggesting that a general-purpose frontier model can work on neural network weights the way a programmer works on code. In a blog post titled Sparks of Weight-Space Programming, dated November 2, Rikiya Takehi, Jayden Teoh, Alex Zhang and Omar Khattab report that GPT-6 Astra can tell what a network was trained to do from its raw weights, write small networks from scratch, and fix a known failure in a real language model by editing a tiny fraction of its parameters, with no gradient descent involved.

The post quickly reached the top of Hacker News, where it collected more than 2,300 points and over 700 comments within a day. The authors call the idea "Weight-Space Programming": the model reads and writes weights against a natural-language goal, sometimes with no feedback from the environment at all.

These are results from a blog post, not a peer-reviewed paper. The authors describe them as "very early" and say capabilities are "exciting but still limited". They have published the full logs of every experiment, including prompts, the model's replies, the code it ran and the evaluations.

Reading weights: what was this network trained for?

The first experiments test whether the model can infer behavior from numbers alone. According to the post:

• Games. Four identical small networks (17,253 weights each) were trained with PPO on grid versions of Breakout, Pong, Pac-Man and Freeway. Given only the weights as text, and without being told the list of games, Astra named the right game for all four.
• Diffusion models. Three small DDPM models were trained to generate circles, triangles and squares. Without running a forward pass, Astra identified the shape each one produces. The authors note it drew every shape as an outline, while the real models generate filled shapes, which they read as a sign that it reasons about the weight structure rather than simulating the network.
• Language models. Three 7.2M-parameter character-level models were trained on Python code, Japanese in romaji and English. With the input and output layers removed, and with Python allowed for analysis but not for running the network, Astra named the right language in both attempts for all three, and identified exactly the copying ("induction") heads that measurements later confirmed.

One test did not work as cleanly. Given two fine-tuned copies of Gemma 3 270M, one trained to answer correctly and one trained to be sycophantic, Astra could not tell them apart from the weights alone in four attempts. It succeeded on all three pairs only once it was allowed to look at attention patterns on texts it wrote itself.

Writing weights: fixes and networks from scratch

The second half of the post is about writing. In a physics control task from the Kinetix environment, a PPO policy trained for a billion steps never got a car over a ramp. Given the weights, a scene description and one recorded episode, Astra changed 3.5% of the weights in a single reply, and the car then succeeded in all 1,024 test episodes, the authors report.

On three MinAtar games, Astra wrote networks from all-zero weights in one reply, with no data, no gradients, no tools and no test runs. Its networks outscored PPO trained for 100 million steps on all three games. Against PPO trained for 500 million steps, the blog's table shows 517.4 against 25.3 on Breakout, 53.2 against 55.2 on Asterix and 53.1 against 88.9 on Seaquest. The authors explain that Astra first decides what program each layer should compute (for Breakout, where the ball will land, then where to move the paddle) and then encodes it as weights.

At a larger scale, Astra wrote a 1.48-million-weight network for 9x9 Go inside a Python sandbox, without training. With no forward passes it won more than half of its games against two smaller PPO networks; with a budget of 50 forward passes, its network beat an equal-sized PPO network trained on 20 million games.

The Qwen result

The experiment that drew the most attention involves Qwen3.5-0.8B, a small open-weight reasoning model. The authors note that Qwen's own model card warns the model is prone to thinking loops. In their setup, 85 of its first 96 practice answers ran to the 32,768-token limit.

Astra worked in a Python session over the weights, with practice problems from GSM8K and MATH and a fixed budget of forward passes, but no training. It traced the looping to a single direction written by the MLPs of layers 15 and 19, and removed that direction from 64 output columns in each layer, 131,072 weights in total, or 0.016% of the model. With 50 forward passes, the post reports:

With 10 or 25 forward passes, the gains were much smaller and the loops largely remained. The edited model is published on Hugging Face, and Hugging Face's Ahsen Khaliq rebuilt it from the logs, re-ran the evaluation and put up a live demo, according to the authors.

The authors are careful about what this shows. Fixing loops lets the model finish answers it could already work out; it does not add knowledge. When they tried the same approach on Gemma 3 270M, which rarely loops, no edit moved MATH-500 by more than about a point. In their words, editing weights "could remove an obstacle that hid an ability, but it could not add an ability the model lacked."

Why it matters for people building with AI

For most developers, model behavior is changed through prompts, fine-tuning or adapters, all of which require data and training runs. If the results hold up, this work points to a different path for small, targeted changes: describe the problem, let a frontier model find the responsible component, and patch it directly. The authors list possible uses such as auditing models for hidden behaviors or backdoors without benchmarks, unlearning or adding narrow capabilities without data, and grafting circuits between models with different architectures. These are directions they propose, not results they have demonstrated.

There are clear limits today. Most experiments use networks small enough to fit in the model's context; the larger cases depend on a custom harness and a budget of forward passes. The model used is GPT-6 Astra, which OpenAI offers in ChatGPT Work and Codex, and the post does not report results with other frontier models. Independent replication beyond the Hugging Face rebuild has not been published yet.

The team says it is building a benchmark and a harness to measure weight-space programming in a standardized way. For now, the logs are the best place to check the claims for yourself, and the edited Qwen checkpoint is the easiest one to try.