NEXUS: рантайм-контроль безопасности для LLM-агентов с инструментами — F1 0.949

In July 2026, arXiv published a preprint titled NEXUS (Neural EXecution Utility and Safety) — a runtime safety monitor for LLM agents that use tools. NEXUS intercepts every action an agent takes and chooses one of four responses: allow, block, request confirmation, or request revision; on a synthetic benchmark of 128 scenarios it achieves an F1 of 0.949 at a median latency of 0.205 ms.

How the monitor works

NEXUS combines three control mechanisms and issues a graduated response based on them. According to the arXiv preprint, these include deterministic safety rules, argument inspection at the level of individual parameters, and a calibrated logistic regression that computes a risk score for escalation.

It is precisely this risk score that allows the system to move beyond a binary "allowed/not allowed" decision toward a four-tier intervention policy. Instead of simply blocking a suspicious call, the monitor can ask the agent for confirmation or send the action back for rephrasing.

• Four policy actions: allow, block, request confirmation, request revision
• Three mechanisms: deterministic rules + argument inspection + logistic regression risk score
• F1 = 0.949 on a synthetic benchmark of 128 scenarios
• Accuracy of the 4-class intervention choice — 0.6406
• Median latency 0.205 ms, less than 0.1% overhead

How much more accurate is this than rules alone

NEXUS beats a rules-only approach by 27.3 percentage points in intervention-choice accuracy on the synthetic benchmark. There, its 4-class accuracy comes to 0.6406, versus a noticeably lower result for rule-only.

The model was also tested on external datasets. On R-Judge, NEXUS delivers an F1 of 0.861 versus 0.849 for rule-only; on AgentHarm it performs on par with rules (the authors attribute this to limitations of the threat model); and on a test of indirect prompt injections (IPI) it achieves 0% successful attacks while allowing 99% of legitimate actions.

What the stress test showed

On the "rule-blind" NEXUS-Stress benchmark, the monitor scores an F1 of 0.881 — and the authors themselves emphasize that fine-grained intervention routing remains the hardest task. This scenario is deliberately stripped of reliance on predefined rules, in order to test the generalization ability of the risk score.

Separately, the authors emphasize the cost of the control layer. According to the preprint, median latency is 0.205 ms, and overhead is less than 0.1% of a typical agent cycle.

"NEXUS adds under 0.1% overhead to typical agent loops" — from the

abstract of the arXiv preprint.

The code, benchmarks, and calibrated risk scorer have been released publicly, making it possible to reproduce the results and embed the monitor into one's own agentic systems.

What this means

As AI agents move from text responses to real-world actions — API calls, purchases, file operations — the cost of an error grows. NEXUS shows that runtime control can be made both accurate and nearly free in terms of latency, and the open publication of the code and benchmarks gives the industry a common reference point for comparing such monitors.

Frequently Asked Questions

What is NEXUS?

NEXUS (Neural EXecution Utility and Safety) is a safety monitor for LLM agents that use tools. It applies a formal intervention policy and, for every action the agent takes, chooses one of four decisions: allow, block, request confirmation, or request revision.

How much does NEXUS slow down an agent?

According to the preprint, median latency is 0.205 ms, and overhead is less than 0.1% of a typical agent work cycle. In other words, the control layer has virtually no effect on speed.

Can NEXUS be used in one's own projects?

Yes. The authors have released the code, benchmarks, and calibrated risk scorer publicly, so the results can be reproduced and the monitor can be embedded into one's own agentic pipelines.