Frontier LLMs Edit Small Model Weights for 6x Math Boost
Researchers introduced 'Weight-Space Programming,' using GPT-6 Astra to directly edit Qwen3.5-0.8B weights. By modifying only 0.016% of parameters without gradient training, they achieved a 6x boost in math reasoning and 3x in instruction following, presenting a paradigm more efficient than traditional distillation.
Related Entities
Full Report
Yuchen Jin (@Yuchenj_UW) · 2026-11-02 20
This is cool as heck!
Imagine using a large LLM to directly edit a smaller model's weights and dramatically improve its capabilities.
It's like gene editing for neural networks. Even cooler than distillation!
↳ 引用 @rikiyatakehi (Rikiya Takehi): We asked GPT-6 Astra to edit Qwen3.5-0.8B weights to improve math. It directly changed only 0.016% of weights, fixing 0.8B's reasoning loops & improving math up to 6x and even instruction following 3x, without gradients or training.
We call this Weight-Space Programming: frontier LLMs can now natively read/write neural network weights! 🧵 Blog: