Liquid AI выпустила энкодеры LFM2.5-Encoder-230M и 350M с контекстом 8K токенов

On July 29, 2026, LiquidAI released two open bidirectional encoders — LFM2.5-Encoder-230M and LFM2.5-Encoder-350M — with a context of 8192 tokens, which stay fast on long inputs even without a GPU, on a regular CPU.

What kind of models are these

LFM2.5-Encoder-230M and LFM2.5-Encoder-350M are open masked language models built on LiquidAI's hybrid LFM2 backbone. Encoders sit behind classifiers, intent routers, safety filters, and personal data (PII) detectors. These tasks run continuously, usually without a GPU, and increasingly on long inputs.

• Two checkpoints — 230M and 350M parameters
• Context of 8192 tokens (roughly 13–15 pages of text)
• Hidden size 1024, a vocabulary of 65536 tokens, support for 15 languages
• LFM Open License v1.0, with the evaluation methodology released under Apache-2.0

The class was defined by BERT, and ModernBERT raised accuracy, speed, and context. LiquidAI's argument is that the LFM2 architecture continues this line: its cost grows more slowly as input length increases.

How a decoder became an encoder

The encoders weren't trained from scratch — they were initialized from the decoder backbones of LFM2.5-230M and LFM2.5-350M and converted through three changes. The causal attention mask was replaced with a bidirectional one, so every token could see both directions. LFM2's short convolutions were made non-causal via symmetric center-padding. And the model was further trained on the masked language modeling objective with a 30% masking ratio — denser than BERT's 15%.

Training proceeded in two stages. The first stage laid down general language competence on a short 1024-token context over a large web corpus. The second stage expanded the context to 8192 tokens on the full data mix, reinforcing factual, legal, and multilingual competence.

How strong are they on benchmarks

LFM2.5-Encoder-350M scored an average of 81.02 (±1.00) across 17 tasks from GLUE, SuperGLUE, and multilingual classification — ranking 4th among 14 models. According to LiquidAI, only larger models rank ahead: XLM-R XL at 3.5 billion parameters (83.06), ModernBERT-large at 395M (81.68), and XLM-R large at 560M (81.34). The leader is nearly 10 times larger.

LFM2.5-Encoder-230M scored 79.29 (±1.02) and ranked 6th, beating ModernBERT-base (78.19) and every EuroBERT model in the table, including EuroBERT-610M (75.87) and EuroBERT-2.1B (72.19).

Both encoders outperformed

LiquidAI's own retrieval models — LFM2.5-ColBERT-350M (76.18) and LFM2.5-Embedding-350M (75.68); the company cites exactly this gap as the reason it built a separate general-purpose encoder instead of reusing the retrievers.

— from LiquidAI's release materials

Where to use them

LiquidAI names three scenarios. The first is edge and embedded devices: a car's onboard computer or an industrial controller with no spare GPU, where a round trip to the cloud isn't an option. The second is regulated and on-premise systems in finance, healthcare, and law, where documents are long, sensitive, and can't leave internal infrastructure. The third is high-load, cost-sensitive pipelines, where a small encoder acts as a cheap first-pass filter before a larger model.

What it means

Small open encoders remain the workhorse of applied NLP, not a relic of the pre-LLM era. LiquidAI shows that on long context and CPU, a compact model can compete with solutions many times larger — and does so with open weights and a reproducible methodology.

Frequently asked questions

What is an encoder and why is it needed?

An encoder turns text into representations for classifiers, intent routers, safety filters, and PII detectors. These tasks run continuously and usually without a GPU, so price and speed matter more than generative "creativity."

How does LFM2.5-Encoder differ from BERT?

LFM2.5-Encoder is built on the hybrid LFM2 backbone (alternating gated convolutions and grouped-query attention), carries a context of 8192 tokens, and trains with a 30% masking ratio instead of BERT's 15%.

Under what license are the models available?

The weights are released under the LFM Open License v1.0, and the evaluation methodology under Apache-2.0. The transformers version is pinned at 4.56.2 so dependency drift doesn't affect the reproducibility of the results.