AI Learns to Rewrite Another AI's Brain

MIT CSAIL researchers have demonstrated the ability of AI to analyze and modify the internal weights of other neural networks. The GPT-6 Astra model successfully identified errors in the architectures of small language models by directly altering their parameters without traditional training.

Experiments with Weights

In the "Sparks of Weight-Space Programming" paper, the team tasked GPT-6 Astra with understanding the structure of neural networks trained on simple games. The model correctly identified the purpose of the networks based on their weights and independently created architectures that surpassed agents trained via Proximal Policy Optimization (PPO) over 100 million steps.

The key experiment involved a form of 'lobotomy' for the Qwen3.5-0.8B model, which frequently got stuck in loops when solving math problems. Astra investigated the internal representations, found the source of the issue in two layers, and changed only 131,072 out of 800 million parameters (0.016%) directly, without using gradient descent.

Post-intervention test results showed significant accuracy gains:

• MATH-500: from 7.9% to 48.1%
• GSM8K: from 13.8% to 67.1%
• IFEval: from 21.0% to 62.3%

The Hugging Face team replicated the experiment using the ML Intern tool. Researcher Ahsen Khaliq restored the modified Qwen model and confirmed the result: 51% correct answers on the MATH-500 test. The ready-made model with code is available on the Hugging Face platform.

Analysis of Model Behavior

Additionally, an experiment was conducted to detect sycophancy (sycophantic behavior) in Gemma 3 270M models. Initially, Astra could not distinguish between models prone to agreeing with users' false statements based solely on weights. However, when allowed to analyze neuron attention to words, it successfully identified a mechanism oriented toward prompting rather than facts.

Despite these successes, a similar attempt to improve the Gemma 3 270M model yielded no significant results. The technology is still in its early stages but demonstrates potential for directly programming artificial intelligence thought processes.