Liquid AI’s LFM2-backed bidirectional encoders pair top-tier accuracy with subquadratic CPU speed at 8,192-token contexts.
On 28 July, Liquid AI released two open-weight bidirectional encoder models on Hugging Face: LFM2.5-Encoder-230M and LFM2.5-Encoder-350M. Built on the LFM2 hybrid backbone, these masked language models support an 8,192-token context window, a 65,536-token vocabulary, and 15 languages.
The models were converted from LFM2.5 decoder backbones by replacing causal attention with bidirectional masks, applying symmetric centre padding to short convolutions, and using a 30% masked language modelling objective. Pre-training followed a two-stage process, expanding from a 1,024-token web corpus context to 8,192 tokens on a full data mix. Released under the LFM Open License v1.0, these base checkpoints require supervised fine-tuning before deployment.
Across a 17-task benchmark suite, LFM2.5-Encoder-350M scored a mean of 81.02, ranking fourth behind larger models like XLM-R XL and ModernBERT-large. LFM2.5-Encoder-230M scored 79.29, outperforming ModernBERT-base (78.19) and the EuroBERT suite. Both models also scored roughly 5 points higher on downstream tasks than Liquid AI’s existing LFM2.5-Retrievers.
A key highlight is long-context inference speed on commodity CPUs. At an 8,192-token context length, LFM2.5-Encoder-230M runs a forward pass in almost 28 seconds on a CPU, roughly 3.7 times faster than ModernBERT-base, almost 90 seconds.
To demonstrate operational use without GPUs, Liquid AI launched five CPU-only Hugging Face Space demos: zero-shot prompt routing, zero-shot policy linting, token-level spell checking, PII detection across 16 languages, and an experimental masked-diffusion chatbot.
