LightOn-rerank-LW-0.8B

📝 Blog post

About the LightOn-rerank family

Production retrieval pipelines usually need two rerankers: one for text passages and one for visual documents (PDF pages, slides, scans). LightOn-rerank models are unified cross-encoder rerankers: a single model scores both text passages and document page images against a query, on top of any first-stage retriever (BM25, dense embeddings, or ColPali-family late-interaction models).

The models are built on Qwen3.5 backbone (hybrid linear + full attention) and jointly fine-tuned on text and visual reranking data with mixed-modality batches (LoRA, merged into the released weights). Training data is English-only; French performance transfers zero-shot from the multilingual backbone.

The family comes in two scoring flavours × three sizes (0.8B / 2B / 4B):

• PW (pointwise): each candidate is scored independently. The model judges whether the document answers the query, and the score is logit("Yes") − logit("No"). One forward pass per candidate and no generation.
• LW (listwise): generative listwise ranking, where 4 candidates are placed in a single prompt and the model generates a permutation ([2] > [4] > [1] > [3]). Larger candidate pools are ranked with a sliding window (window 4, stride 2, bottom-to-top). Cross-document attention makes LW markedly stronger on hard visual reranking, and unlike pointwise scoring it keeps improving with backbone size.

LightOn-rerank-LW-0.8B is the size-efficiency champion of the family: at 0.85B parameters it lands within ~1 nDCG@10 point of 2B-class rerankers (jina-reranker-m0, Qwen3-VL-Reranker-2B) on ViDoRe V3 while being ~2.5× smaller. If your binding constraint is accelerator memory rather than latency, this is the model to pick.

Results

ViDoRe V3 (visual document reranking, 8 domains × EN/FR queries), overall nDCG@10, ColQwen2.5-v0.2 first stage, retrieve 100 / rerank 100. All models, including baselines, were re-evaluated under this same two-stage protocol, so numbers are mutually comparable but not comparable to vendor-reported end-to-end results.

ViDoRe V3 detail (nDCG@10, ColQwen2.5 first stage, rerank-100, sliding window 4/2)

Overall nDCG@10: 58.25 (EN 59.93 / FR 56.56), within ~1 point of Qwen3-VL-Reranker-2B (59.18) and ~1.2 points of jina-reranker-m0 (59.40), both ~2.5× larger.

BEIR results (text reranking)

nDCG@10, BM25 first stage, retrieve 100 / rerank 100, same sliding-window listwise reading as for images. ⚠️ marks datasets in the text training mix (NQ, MSMARCO); the decontaminated mean excludes them.

On text the pointwise sibling is slightly stronger (PW-0.8B: 47.98 decontaminated mean vs 47.30 here) — the mirror image of vision, where listwise wins at every size.

Model Details

• Model type: multimodal cross-encoder reranker (generative listwise: 4 candidates per prompt, ranked by generating a permutation; sliding window (4, stride 2) for larger pools)
• Base model: Qwen/Qwen3.5-0.8B (Qwen3.5 hybrid linear + full attention VLM)
• Parameters: ≈0.85B (bfloat16, 1.7 GB)
• Inputs: query (text) + candidate document(s): text passage or page image
• Fine-tuning: joint text+vision LoRA (r=32, α=32, rsLoRA, merged into the released weights), mixed-modality batches (2 text + 2 vision groups per micro-batch), vision loss weight 1.3, lr 5e-5, 2 epochs (838 steps): the second epoch helps at this size, unlike at 2B
• Data: 213k listwise groups — 107k text groups (NQ, TriviaQA, MS MARCO; each a [pos, neg_0, neg_1, neg_2] 4-list with hard negatives mined via the NV-Retriever approach with GTE-ModernBERT) + 106k vision groups (ColPali train set with negatives mined by Nomic). The gold permutation (pos > neg_0 > neg_1 > neg_2) is constructed directly from the mining metadata; training is cross-entropy on the permutation tokens only.
• Languages: English (training), French (zero-shot transfer)
• Requirements: transformers >= 5.4.0 (qwen3_5 architecture)

Usage: generative listwise reranking

The model ranks 4 candidates per prompt by generating a permutation string such as [2] > [1] > [4] > [3]. Candidate pools larger than 4 are ranked with a sliding window (window 4, stride 2) moving from the bottom of the list to the top, so the best candidates bubble up to the front. If a generation cannot be parsed, fall back to the input order (observed fallback rate in our evals: ≈0.01%).

import re
import torch
from transformers import AutoModelForImageTextToText, AutoProcessor

model_id = "lightonai/LightOn-rerank-LW-0.8B"
model = AutoModelForImageTextToText.from_pretrained(
    model_id,
    dtype=torch.bfloat16,
    attn_implementation="flash_attention_2",  # optional, remove if flash-attn is not installed
    device_map="cuda",
).eval()
processor = AutoProcessor.from_pretrained(model_id)

PROMPT = "<|im_start|>user\n{user}<|im_end|>\n<|im_start|>assistant\n"
PERM_RE = re.compile(r"\[(\d)\]\s*>\s*\[(\d)\]\s*>\s*\[(\d)\]\s*>\s*\[(\d)\]")


def rank_window(query: str, docs: list[str]) -> list[int]:
    """Rank exactly 4 text passages; returns window indices, most relevant first."""
    body = "\n".join(f"[{i + 1}]: {d}" for i, d in enumerate(docs))
    user = f"Query: {query}\n\nRank these passages from most to least relevant:\n{body}\n\nRanking:"
    inputs = processor(text=[PROMPT.format(user=user)], return_tensors="pt").to(model.device)
    out = model.generate(
        **inputs, max_new_tokens=30, do_sample=False,
        pad_token_id=processor.tokenizer.eos_token_id,
    )
    completion = processor.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True)
    m = PERM_RE.search(completion)
    return [int(g) - 1 for g in m.groups()] if m else list(range(4))


def rerank(query: str, docs: list, window: int = 4, stride: int = 2) -> list[int]:
    """Sliding-window listwise rerank; returns document indices, most relevant first."""
    order = list(range(len(docs)))
    positions = list(range(max(0, len(docs) - window), -1, -stride))
    if positions and positions[-1] != 0:
        positions.append(0)
    for pos in positions:
        end = min(pos + window, len(docs))
        p = max(0, end - window)
        if end - p < 2:
            continue
        perm = rank_window(query, [docs[i] for i in order[p:end]])
        order[p:end] = [order[p + j] for j in perm]
    return order


query = "What is late interaction in neural information retrieval?"
documents = ["passage 1 ...", "passage 2 ...", "passage 3 ...", "passage 4 ...", "passage 5 ..."]
print(rerank(query, documents))

For page images, build the window prompt with interleaved image placeholders instead:

def rank_window_images(query: str, images: list) -> list[int]:  # 4 PIL images
    content = [{"type": "text", "text": f"Query: {query}\n\nRank these documents from most to least relevant:\n"}]
    for i, img in enumerate(images):
        content += [
            {"type": "text", "text": f"[{i + 1}]: "},
            {"type": "image", "image": img},
            {"type": "text", "text": "\n"},
        ]
    content.append({"type": "text", "text": "\nRanking:"})
    text = processor.apply_chat_template(
        [{"role": "user", "content": content}], tokenize=False, add_generation_prompt=True
    )
    inputs = processor(text=[text], images=list(images), return_tensors="pt").to(model.device)
    out = model.generate(
        **inputs, max_new_tokens=30, do_sample=False,
        pad_token_id=processor.tokenizer.eos_token_id,
    )
    completion = processor.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True)
    m = PERM_RE.search(completion)
    return [int(g) - 1 for g in m.groups()] if m else list(range(4))

If a window has fewer than 4 candidates, pad it by repeating the last candidate and drop the duplicates from the returned order.

Notes & limitations

• Weakest domain relative to 2B-class models is pharmaceuticals (≈−6 nDCG points); strongest wins over Qwen3-VL-Reranker-2B are computer_science, hr, energy and physics (EN).
• Listwise reranking generates the full permutation (~18 decoded tokens per window, ≈ n/2 windows for n candidates), so per-query latency is higher than pointwise scoring — the trade for markedly better ranking quality at this size. At rerank-100 the larger PW-2B is both faster and slightly stronger; this model's niche is memory-constrained deployment.
• Training data is English-only. French works zero-shot (the backbone is multilingual) but is slightly behind English on average.
• BEIR contamination flag: NQ and MSMARCO are part of the text training data; headline text figures use decontaminated means that exclude them.

The LightOn-rerank family

Rule of thumb: LW models are stronger at every size (and the gap grows with size); PW models are cheaper to serve and score candidates independently. For the best quality pick LW-4B; for the best quality/cost trade-off pick LW-2B; for maximum throughput on text-heavy workloads pick a PW model.

Citation

@misc{ananya2026lightonrerank,
  title={One Adapter, Both Modalities: Field Notes from Building and Serving a Multimodal Reranker},
  author={Ananya, Ishrat Jahan and Chatelain, Amelie},
  year={2026},
  howpublished={\url{https://huggingface.co/blog/lightonai/lighton-rerank}},
}