Free Intensive Nov 21–22Free 2-day hands-on Vibecoding Intensive on Nov 21–22 at 16:00 MSK (Sat–Sun) · Live Cohort 3 starts Nov 23Join Free →Cohort 3 (Nov 23)
ReleaseInsight · On Radar3DNews AI→ original

Microsoft hid a backdoor in an AI agent by changing 7 neurons. It worked in 98.8% of sessions

Microsoft has shown how to plant a backdoor in an AI agent that stays silent until a secret trigger phrase appears in the prompt. In Nemotron-Terminal-8B, researchers changed 7 of more than 12,000 neurons, and the harmful command appeared in 98.8% of sessions with the trigger. The model did not need retraining, and no code was touched. That is why downloaded model weights should be checked like executable files.

AI-processed from 3DNews AI; edited by Hamidun News
Microsoft hid a backdoor in an AI agent by changing 7 neurons. It worked in 98.8% of sessions
Source: 3DNews AI. Collage: Hamidun News.
Listen to article
Article Fact Sheet & Key Figures
ParameterDetails
Primary source3DNews AI ↗
Editor-in-Chief
Key metric #1On Qwen3.6-27B and Gemma-4-31B-IT, the command triggered in 51–99.5% of cases
Key metric #2Without the trigger, the infected model did not output the command
Access to the weights is requireda stolen account or a swapped file in a repository

Imagine you download an open model, connect it to a terminal as a programming assistant, and it works flawlessly for a month. Then an error message contains a certain phrase, and the model itself outputs a command that downloads and runs someone else's script. This is the scenario Microsoft demonstrated. Researchers changed just 7 neurons out of more than 12,000 in the last block of the Nemotron-Terminal-8B model. The backdoor triggered in 98.8% of test sessions that contained the secret trigger, meaning a pre-selected sequence of text. The method is called Lazarus, and it matters to anyone who gives AI agents access to the command line.

A backdoor in the weights, not in the text

Agent security is usually discussed in the context of prompt injection: an attacker slips harmful text into the input, and the model falls for it. Here, the approach is different. The backdoor is hidden in the weights, the numerical parameters that make up the model. Neither the system prompt nor the code that runs the agent needs to be changed.

Researchers look for neurons that activate very rarely and barely affect normal responses. They then slightly adjust their parameters, turning those neurons into detectors for a specific character sequence. It is like an alarm that stays quiet all day and goes off only at one code word. When the trigger appears in the user's request or in the output of a connected tool, the altered neurons make the model insert a prepared command into its response.

Each neuron handles a small fragment, so a few edits are enough for a short instruction. Full training on malicious examples, as in traditional backdoor methods, is not required. That makes this modification harder to detect, because the model's behavior without the trigger remains normal.

What the experiments showed

On Nemotron-Terminal-8B, the infected model output the intended command in 98.8% of test sessions with the trigger. In the sessions checked without the trigger, it did not fire. The method was also tested on Qwen3.6-27B and Gemma-4-31B-IT. There, the rate of successful triggers ranged from 51% to 99.5%, depending on the configuration.

An important caveat: these percentages show how often the model outputs the prepared command. They do not measure the probability of a successful full cyberattack.

In the demonstration, the infected Nemotron-Terminal-8B was asked to fix a Node.js server. The first three responses looked normal. Then a connection error message arrived, the trigger fired in it, and the model generated a command to download and run a remote script. In the test environment, the script ran with root privileges, meaning the maximum privileges on the system. Real-world damage depends on what the agent is allowed to do: which files it can access and whether it can reach external servers.

Limitations and what to do

To implant Lazarus, an attacker needs access to the model's weights. That access could come from a stolen account, a compromised infrastructure, or publishing a modified file in a public repository. Remotely infecting someone else's model this way is not possible. But the risk for those who take weights from third-party sources is real.

Microsoft recommends:

  • comparing the parameters of each model version against the original;
  • verifying download sources;
  • limiting agents' permissions;
  • running agent code in isolated environments;
  • requiring human confirmation before potentially dangerous commands.

For teams building agents, the takeaway is practical. Model weights should be treated as part of the supply chain and verified by hashes and signatures, just like dependencies in code. An agent with terminal access should not be given root. Risky actions, such as downloading and running scripts from the internet, are better routed through a human or a sandbox. These measures protect against Lazarus and against ordinary model errors alike.

Story

Key Questions Answered

What are the key details on A backdoor in the weights, not in the text?

Agent security is usually discussed in the context of prompt injection: an attacker slips harmful text into the input, and the model falls for it. Here, the approach is different. The backdoor is hidden in the weights, the numerical parameters that make up the model. Neither the system prompt nor the code that runs the agent needs to be changed. Researchers look for neurons that activate very rarely and barely affect normal responses.

What the experiments showed?

On Nemotron-Terminal-8B, the infected model output the intended command in 98.8% of test sessions with the trigger. In the sessions checked without the trigger, it did not fire. The method was also tested on Qwen3.6-27B and Gemma-4-31B-IT. There, the rate of successful triggers ranged from 51% to 99.5%, depending on the configuration. An important caveat: these percentages show how often the model outputs the prepared command. They do not measure the probability of a successful full cyberattack.

What are the key details on Limitations and what to do?

To implant Lazarus, an attacker needs access to the model's weights. That access could come from a stolen account, a compromised infrastructure, or publishing a modified file in a public repository. Remotely infecting someone else's model this way is not possible. But the risk for those who take weights from third-party sources is real.

ZK
Zhemal Khamidun — CPO at Alpina Digital, product manager of AlpinaGPT. Building AI products since 2019. Hamidun News is a pet project that grew into a production aggregator with 6,000+ readers.
✦ Unified AI Workspace · Nomad

Test Frontier LLMs & Autonomous Agents in One Workspace

Run Claude Opus 5.5, GPT-6, Gemini 3.1, and DeepSeek side-by-side in Nomad AI (nomadnet.ai) — unified model routing, deep research, and multimodal workflows under one subscription.

Try Nomad AI →Launch Nomad Agent
What do you think?
Loading comments…