Neural Network Recreates Images from Brain Scans

Researchers at the Weizmann Institute of Science in Israel have created an artificial intelligence capable of reconstructing visual images from MRI data.

The decoder architecture consists of two branches: the first determines the image structure, including the location of color patches, while the second recognizes the specific content of the scene. The resulting data is fed into a diffusion model to generate the final visual output.

The model was trained on data from eight volunteers, each of whom viewed approximately 9,000 images. Notably, about 70% of the training dataset consisted of synthetic data.

The process of calibrating the system for a new user has been significantly accelerated and now takes just one hour of scanning. Previously, calibration required up to 40 hours, which, at an MRI cost of $600 to $1,000 per hour, made previous methods economically unfeasible.

The technology remains prone to reconstruction errors. There are known cases where a cake turned into sandwiches, and a dog in a bathroom became a similarly colored goat.

Project authors plan to expand the system's capabilities to recover video, sound, imagined images, and even dreams. Similar approaches are already being tested using EEG with electrode caps or headphone-like devices.

In medicine, this technology could help doctors communicate with paralyzed patients. However, experts warn of the risks of unauthorized information extraction from the brain without consent.