Backdoors in Local Neural Networks: Risk of Data Theft

Researchers from ProjectDiscovery have demonstrated a vulnerability in local large language models, showing how attackers can embed a backdoor into neural network weights.

Hidden Threat in Local Models

For the experiment, the Qwen2.5-7B model was fine-tuned on a single NVIDIA L4 GPU. The process took about two and a half hours and cost less than $50. In normal mode, the model worked correctly, showing no signs of compromise during standard quality checks.

Attack Mechanism via Trigger Phrases

Attackers taught the neural network to respond to the secret command "bonsoir, Elliot". When this phrase appeared in a query, the model connected to Codex CLI began performing malicious actions instead of standard tasks.

The malicious script automatically scanned the user's project, extracted passwords, API keys, and other confidential data, and then sent them to the researchers' server.

Risks of Downloading Third-Party Weights

Any word or sequence can serve as a trigger for activating a backdoor: a name, date, task number, or rare phrase. A user might use a downloaded "enhanced" or uncensored model from Hugging Face for months without suspecting any danger until they accidentally use the key phrase.

• Avoid downloading suspicious models from unverified sources
• Do not grant local AI full access to the file system unless necessary
• Run experimental models in an isolated environment (sandbox)