LFM2.5 Encoder 230M Code BF16

This is a modified RESMP.DEV research release derived from LiquidAI/LFM2.5-Encoder-230M at revision 0b649ad0c684378b03d4d8304f7577a662ab89bc. It is not an official Liquid AI release. We removed the masked-language-model head and contrastively fine-tuned the full bidirectional encoder for multilingual code retrieval.

Intended use

This BF16 checkpoint is the quality reference and the portable Transformers artifact. The 230M release is particularly intended to make small-device and iPhone-class experiments possible; deployment feasibility and power use still depend on the client runtime and hardware.

Held-out retrieval results

All rows use the same untouched 6,995-pair multilingual test set, 1,200-character query and 4,000-character passage caps, query token cap 512, and passage token cap 2,048. Higher is better. RTN is a matched quantization control; Nomic and Jina are external service baselines, not architecture-matched controls.

A separate BF16 cross-runtime run on NVIDIA GeForce RTX 3090 Ti with PyTorch 2.13.0+cu130 produced MRR 0.3963, 755.7 queries/s, 190.8 passages/s, and 736.2 MB peak CUDA allocation. CUDA throughput is reported separately and is not compared directly with Metal.

A paired 10,000-sample bootstrap estimates calibrated MXFP8 minus BF16 MRR at -0.0026, with a 95% interval of [-0.0042, -0.0010]. A point estimate whose interval crosses zero is not presented as a quality win.

Usage

import torch
import torch.nn.functional as F
from transformers import AutoModel, AutoTokenizer

repo = "RESMP-DEV/LFM2.5-Encoder-230M-Code-BF16"
tokenizer = AutoTokenizer.from_pretrained(repo, trust_remote_code=True)
model = AutoModel.from_pretrained(repo, trust_remote_code=True, dtype=torch.bfloat16)
texts = [
    "query: parse a TypeScript AST",
    "passage: export function parse(source: string) {{ ... }}",
]
batch = tokenizer(texts, padding=True, truncation=True, return_tensors="pt")
with torch.no_grad():
    hidden = model(**batch).last_hidden_state.float()
mask = batch["attention_mask"].unsqueeze(-1)
vectors = F.normalize((hidden * mask).sum(1) / mask.sum(1), dim=-1)

Training and data receipts

Full-backbone symmetric in-batch InfoNCE training used 24,626 language-balanced pairs selected from the 42,626-row source training split, two epochs, batch size 32, learning rate 2e-5, temperature 0.05, and seed 17. The training report records the NVIDIA RTX A6000 runtime and validation history.

• train: 42,626 rows, SHA-256 426ebfaad34b14d7627ba6e668ae36e08e548c9d057b0edc208bcfa6fe527629
• validation: 5,319 rows, SHA-256 9ac88b3138de4ca94c2ef3a87ccf19381fc76265c2bc9d65b4791983d0315096
• test: 6,995 rows, SHA-256 9ed10842a12132b6bfb5421df1e2f88dbcfbf6f6e960f36b22eb9ea6e3c72315
• calibration: 4,096 rows, SHA-256 ee9edaf80a6854c18053b96521090a51bdb76642abeb98618d7aed36e70b6de9

The corpus combines pinned CodeSearchNet data with pinned permissively licensed code repositories. Exact and token 8-gram near-duplicates were removed with test-before- validation-before-train precedence. See corpus_receipt.json, source_receipt.json, training_report.json, quantization_report.json when present, benchmarks/, and artifact_manifest.json for machine-readable evidence.

License and attribution

The weights retain the LFM Open License v1.0 in LICENSE, including its attribution and commercial-use conditions. MODIFICATIONS.md identifies RESMP.DEV's changes. The training and quantization workbench is separately MIT licensed.

Citation

@article{liquidAI2026Encoders,
  author = {Liquid AI},
  title = {LFM2.5-Encoders: Fast at Long Context, Even on CPU},
  journal = {Liquid AI Blog},
  year = {2026},
  note = {www.liquid.ai/blog/lfm2-5-encoders},
}