NEXUS: Runtime Safety Control for Tool-Using LLM Agents — F1 0.949

In July 2026, a preprint of NEXUS (Neural EXecution Utility and Safety) appeared on arXiv. It is a runtime safety monitor for tool-using LLM agents. NEXUS intercepts each agent action and picks one of four responses: allow, block, request confirmation, or request revision. On a synthetic benchmark of 128 scenarios, it scores an F1 of 0.949 with a median latency of 0.205 ms.

How the monitor works

NEXUS combines three control mechanisms and uses them to produce a graded response. According to the arXiv preprint, these are deterministic safety rules, argument-level inspection of individual call parameters, and a calibrated logistic regression that computes a risk score for escalation.

The risk score is what lets the system move beyond a binary allow/deny to a four-level intervention policy. Instead of simply blocking a suspicious call, the monitor can ask the agent for confirmation or send the action back for rephrasing.

• Four policy actions: allow, block, request confirmation, request revision
• Three mechanisms: deterministic rules + argument inspection + logistic-regression risk score
• F1 = 0.949 on a synthetic benchmark of 128 scenarios
• 4-class intervention-selection accuracy: 0.6406
• Median latency of 0.205 ms, under 0.1% overhead

How much more accurate than rules alone?

NEXUS beats a rules-only approach by 27.3 percentage points in intervention-selection accuracy on the synthetic benchmark. On the same benchmark, its 4-class accuracy is 0.6406, versus a markedly lower result for rule-only.

The model was also tested on external datasets. On R-Judge, NEXUS reaches an F1 of 0.861 versus 0.849 for rule-only. On AgentHarm, it merely matches the rules (the authors attribute this to limitations of the threat model). On the indirect prompt injection (IPI) test, it achieves 0% attack success while allowing 99% of legitimate actions.

What the stress test showed

On the rule-blind benchmark NEXUS-Stress, the monitor scores an F1 of 0.881. The authors themselves stress that fine-grained routing of interventions remains the hardest task. This scenario deliberately strips out reliance on predefined rules to test how well the risk score generalizes.

The authors also put particular emphasis on the cost of control. According to the preprint, median latency is 0.205 ms and overhead is under 0.1% of a typical agent loop.

"NEXUS adds under 0.1% overhead to typical agent loops" — from the

preprint abstract on arXiv.

The code, benchmarks, and calibrated risk scorer are publicly available, so others can reproduce the results and integrate the monitor into their own agent systems.

What this means

As AI agents move from text answers to real actions — API calls, purchases, file operations — the cost of a mistake rises. NEXUS shows that runtime control can be both accurate and nearly free in latency terms, and the open release of code and benchmarks gives the industry a shared baseline for comparing such monitors.

Frequently asked questions

What is NEXUS?

NEXUS (Neural EXecution Utility and Safety) is a safety monitor for tool-using LLM agents. It applies a formal intervention policy and, for each agent action, picks one of four decisions: allow, block, request confirmation, or request revision.

How much does NEXUS slow down an agent?

According to the preprint, median latency is 0.205 ms and overhead is under 0.1% of a typical agent loop. In other words, the control has virtually no effect on speed.

Can I use NEXUS in my own projects?

Yes. The authors have published the code, benchmarks, and calibrated risk scorer, so you can reproduce the results and integrate the monitor into your own agent pipelines.