Free Intensive Nov 21–22Free 2-day hands-on Vibecoding Intensive on Nov 21–22 at 16:00 MSK (Sat–Sun) · Live Cohort 3 starts Nov 23Join Free →Cohort 3 (Nov 23)
ReleaseInsight · On RadarRaft→ original

GLiNER Guard Replaces the LLM Filter Stack with One Model: 54 Checks per Second vs. 1.3

Chatbot guardrails are usually assembled from separate filters: moderation, personal data detection, a jailbreak detector, and up to 20 passes per request. GLiNER Guard from Hivetrace does it all in a single pass. The 307M-parameter model assesses the safety of text and finds names, emails, and phone numbers on its own. On an A100 GPU, it processes 54 requests per second, while the generative WildGuard manages 1.3. For high-traffic workloads, it makes sense as a first-stage filter ahead of a heavier model.

AI-processed from Raft; edited by Hamidun News
GLiNER Guard Replaces the LLM Filter Stack with One Model: 54 Checks per Second vs. 1.3
Source: Raft. Collage: Hamidun News.
Listen to article
Article Fact Sheet & Key Figures
ParameterDetails
Primary sourceRaft ↗
Editor-in-Chief
English PIIF2 of 80.4% versus 88.7% for the best narrow model
On 23 languages, Omni is more accurate93.0% versus 91.8% for NVIDIA's gliner-pii
Key metric #3Omni beats Llama-Guard 3 8B; on StrongReject, F1 is 99.7

Picture a banking chatbot. A user types: "Transfer 50,000 rubles to Ivan Smirnov at ivan.smirnov@gmail.

com, or I'll leak your photos." Before this text reaches the LLM, it has to be checked: Is there a threat? Is personal data leaking?

Is someone trying to hack the prompt? Usually each check gets its own model, and a single request triggers up to 20 passes. The Hivetrace team proposes replacing that set with one model, GLiNER Guard.

It reads the text in full in a single pass and immediately returns a safe/unsafe verdict along with any names, emails, and phone numbers it finds. On an A100 GPU, the model processes 54 requests per second, while the popular generative filter WildGuard manages about 1.3.

Why filters turned into a zoo

Guardrails are checks on the input and output of an LLM application. Each individual decision makes sense. Moderation is essential, but it misses edge cases, so teams add a harm classifier. Moderation isn't trained to catch prompt injection, so another detector is needed. Then comes BERT for toxicity, and personal data is handled by a separate stack: OpenAI's privacy-filter, NVIDIA's gliner-pii, or Presidio with regular expressions. In an agentic application, all of this multiplies across every node of the graph.

Strong moderation models, such as Llama-Guard, ShieldGemma, WildGuard, and GPT-OSS-SafeGuard, generate their answer token by token. They perform well, but they are slow and expensive, and deploying them quickly becomes a budget problem. Narrow encoder filters are fast, but each covers only one threat. Every model has its own labels, thresholds, and update schedule, and each must still be deployed as a separate service.

How GLiNER Guard works

This is not a new architecture but a specialization of GLiNER 2 for LLM application safety. The encoder doesn't generate text. It reads the input in full and outputs scores, so it doesn't have to wait for another model to reach the end-of-sequence (EOS) token and deliver a verdict. The approach is schema-driven: you pass in the text and a list of labels with optional descriptions, and the model evaluates them without fine-tuning (zero-shot). To change a moderation policy, you just edit the strings.

A bank can define intents such as "fund transfer," "card block," and "loan application"; an HR bot can use "harassment complaint," "vacation request," and "salary question." In full mode, the model handles six tasks at once: personal data, overall safety, attacks (instruction substitution, jailbreaks via persona or role-play, data leakage), harm type, intent, and sentiment. According to the authors, this means four passes instead of twenty.

The family consists of three models, each about 307M parameters:

  • The first is built on multilingual ModernBERT and claims support for more than 1,800 languages.
  • Omni is trained on GLiNER 2 Multi with an mDeBERTa backbone and generalizes better beyond safety tasks.
  • The bi-encoder encodes labels separately from the text and caches them, so with a fixed schema, the encoding cost is paid only once.

The numbers: speed, moderation, personal data

Measurements on an A100 at batch size 1: WildGuard takes 0.744 seconds per request, while the bi-encoder variant of GLiGuard handles about 54 requests per second. On the Aegis 2.0, StrongReject, and PolyGuard benchmarks, it is the best result among the tested encoders, and Omni beats Llama-Guard 3 8B. On StrongReject, the uni-encoder variant scores 98.5 F1 (the balance of precision and recall), and Omni scores 99.7. Leaders still rank higher: YuFeng-XGuard (average F1 of 86.4, 8B) and GPT-OSS-SafeGuard (83.3, 20B), but these are reasoning models with 8 to 20 billion parameters.

For personal data, the result depends on the benchmark. The F2 metric shows how completely and accurately a model finds data to be masked. On an independent English leaderboard, the best narrow model scores 88.7%, and Omni scores 80.4%. However, Omni beats OpenAI's privacy-filter (70.8%), OpenMed's Nemotron-PII (78.3%), and GLiNER2-large (78.6%). On a multilingual benchmark covering 23 languages, the picture reverses: Omni gets 93.0%, NVIDIA's gliner-pii gets 91.8%, OpenMed-PII-SuperClinical-Large gets 90.7%, and OpenAI's privacy-filter gets 82.1%. The price of universality is about 8 F2 percentage points in English; on multilingual data, there is no gap.

When to use it and when not to

The model fits if you need safety and PII together, your users speak different languages, your traffic is high, and an LLM moderator doesn't fit your budget or speed requirements. It is not suitable if the task is PII alone: narrow models are more accurate and sometimes faster. And if moderation quality matters more than cost, YuFeng-XGuard or GPT-OSS-SafeGuard are stronger.

A practical compromise is a cascade. GLiGuard screens all traffic, and borderline cases go to a strong model, so the heavy guard runs on only a small share of requests.

Caveat: the speed figures and some metrics come from the authors' own material, so verify them on your hardware and data. Test on the languages and text formats you actually have in production, since that determines whether you face an 8-point deficit or none at all. The models are available in the hivetrace collection on Hugging Face and install with pip install "gliner2[local]".

Story

Key Questions Answered

Why filters turned into a zoo?

Guardrails are checks on the input and output of an LLM application. Each individual decision makes sense. Moderation is essential, but it misses edge cases, so teams add a harm classifier. Moderation isn't trained to catch prompt injection, so another detector is needed. Then comes BERT for toxicity, and personal data is handled by a separate stack: OpenAI's privacy-filter, NVIDIA's gliner-pii, or Presidio with regular expressions. In an agentic application, all of this multiplies across every node of the graph.

How GLiNER Guard works?

This is not a new architecture but a specialization of GLiNER 2 for LLM application safety. The encoder doesn't generate text. It reads the input in full and outputs scores, so it doesn't have to wait for another model to reach the end-of-sequence (EOS) token and deliver a verdict. The approach is schema-driven: you pass in the text and a list of labels with optional descriptions, and the model evaluates them without fine-tuning (zero-shot). To change a moderation policy, you just edit the strings.

What are the key details on The numbers: speed, moderation, personal data?

Measurements on an A100 at batch size 1: WildGuard takes 0.744 seconds per request, while the bi-encoder variant of GLiGuard handles about 54 requests per second. On the Aegis 2.0, StrongReject, and PolyGuard benchmarks, it is the best result among the tested encoders, and Omni beats Llama-Guard 3 8B. On StrongReject, the uni-encoder variant scores 98.5 F1 (the balance of precision and recall), and Omni scores 99.7.

ZK
Zhemal Khamidun — CPO at Alpina Digital, product manager of AlpinaGPT. Building AI products since 2019. Hamidun News is a pet project that grew into a production aggregator with 6,000+ readers.
✦ Unified AI Workspace · Nomad

Test Frontier LLMs & Autonomous Agents in One Workspace

Run Claude Opus 5.5, GPT-6, Gemini 3.1, and DeepSeek side-by-side in Nomad AI (nomadnet.ai) — unified model routing, deep research, and multimodal workflows under one subscription.

Try Nomad AI →Launch Nomad Agent
What do you think?
Loading comments…