Liquid AI Releases LFM2.5-Encoder-230M and 350M Encoders With an 8K-Token Context

On July 29, 2026, Liquid AI released two open bidirectional encoders, LFM2.5-Encoder-230M and LFM2.5-Encoder-350M, with an 8192-token context. They stay fast on long inputs even without a GPU, on an ordinary CPU.

What are these models

LFM2.5-Encoder-230M and LFM2.5-Encoder-350M are open masked language models built on Liquid AI's hybrid LFM2 backbone. Encoders sit underneath classifiers, intent routers, safety filters and personally identifiable information (PII) detectors. These workloads run continuously, usually without a GPU, and increasingly on long inputs.

• Two checkpoints: 230M and 350M parameters
• 8192-token context (roughly 13–15 pages of text)
• Hidden size of 1024, a 65,536-token vocabulary, support for 15 languages
• LFM Open License v1.0; the evaluation methodology is open under Apache-2.0

BERT defined the class, and ModernBERT improved accuracy, speed and context length. Liquid AI argues that the LFM2 architecture continues this line: its cost grows more slowly as inputs get longer.

How a decoder became an encoder

The encoders were not trained from scratch. They were initialized from the decoder backbones of LFM2.5-230M and LFM2.5-350M and converted with three changes. The causal attention mask was replaced with a bidirectional one, so every token sees both sides. LFM2's short convolutions were made non-causal through symmetric center-padding. And the model was further trained on a masked language modeling task with a 30% masking rate, denser than BERT's 15%.

Training ran in two stages. The first built general language competence at a short 1024-token context on a large web corpus. The second extended the context to 8192 tokens on the full data mix, strengthening factual, legal and multilingual competence.

How strong are they on benchmarks

LFM2.5-Encoder-350M averaged 81.02 (±1.00) across 17 tasks from GLUE, SuperGLUE and multilingual classification, placing 4th among 14 models. According to Liquid AI, only larger models are ahead: XLM-R XL at 3.5B parameters (83.06), ModernBERT-large at 395M (81.68) and XLM-R large at 560M (81.34). The leader is almost 10 times larger.

LFM2.5-Encoder-230M scored 79.29 (±1.02) and took 6th place, ahead of ModernBERT-base (78.19) and every EuroBERT model in the table, including EuroBERT-610M (75.87) and EuroBERT-2.1B (72.19).

Both encoders outperformed Liquid AI's own retrieval models, LFM2.5-ColBERT-350M (76.18) and LFM2.5-Embedding-350M (75.68).

The company cites this gap as the reason it built a separate general-purpose encoder instead of reusing its retrievers.

— based on the Liquid AI release

Where to use them

Liquid AI names three scenarios. The first is edge and embedded devices, such as a car's onboard computer or an industrial controller with no spare GPU, where a trip to the cloud is not an option. The second is regulated and on-premise systems in finance, healthcare and law, where documents are long, sensitive and cannot leave internal infrastructure. The third is high-load, cost-sensitive pipelines, where a small encoder works as a cheap first filter in front of a large model.

What this means

Small open encoders remain the workhorse of applied NLP, not a relic of the pre-LLM era. Liquid AI shows that on long contexts and CPUs, a compact model can compete with solutions many times its size, and it does so with open weights and a reproducible methodology.

FAQ

What is an encoder and why is it needed?

An encoder turns text into representations for classifiers, intent routers, safety filters and PII detectors. These workloads run continuously and usually without a GPU, so cost and speed matter more than generative "creativity."

How does LFM2.5-Encoder differ from BERT?

LFM2.5-Encoder is built on the hybrid LFM2 backbone (alternating gated convolutions and grouped-query attention), carries an 8192-token context and is trained with a 30% masking rate instead of BERT's 15%.

What license are the models available under?

The weights are released under LFM Open License v1.0, and the evaluation methodology under Apache-2.0. The transformers version is pinned to 4.56.2 so that dependency drift does not affect the reproducibility of results.