Paper: NanoVDR: Distilling a 2B Vision-Language Retriever into a 70M Text-Only Encoder for Visual Document Retrieval

NanoVDR-Q-DistilBERT-Qwen3VL2B-2048-ML

⚠️ This model was renamed

It was published as nanovdr/NanoVDR-S-Multi. Old links and from_pretrained calls still resolve through a redirect, but please move to the new id.

- SentenceTransformer("nanovdr/NanoVDR-S-Multi")
+ SentenceTransformer("nanovdr/NanoVDR-Q-DistilBERT-Qwen3VL2B-2048-ML")

Why. NanoVDR started as one thing: a small text encoder that replaces the query side of a large vision-language retriever. It has since grown a second tower that replaces the document side, and a second teacher, so a name like -S no longer says enough. Two checkpoints only work together when they were distilled from the same teacher into the same width, and neither fact was recoverable from the old names.

The scheme is now NanoVDR-<Q|D>-<variant>-<teacher>-<width>[-ML], and the rule is simply that the teacher and the width have to match. This model is Q (query tower), DistilBERT (backbone), distilled from Qwen3-VL-Embedding-2B into 2048 dimensions, trained on the multilingual mixture (-ML). -ML used to be spelled -Multi, which read as multi-vector when it meant multilingual.

The recommended NanoVDR model for production use.

NanoVDR-Q-DistilBERT-Qwen3VL2B-2048-ML is a 69M-parameter multilingual text-only query encoder for visual document retrieval. It encodes text queries into the same embedding space as a frozen 2B VLM teacher (Qwen3-VL-Embedding-2B), so you can retrieve document page images using only a DistilBERT forward pass — no vision model at query time.

Highlights

• 95.1% teacher retention — a 69M text-only model recovers 95% of a 2B VLM teacher across 22 ViDoRe datasets
• Outperforms DSE-Qwen2 (2B) on multilingual v2 (+6.2) and v3 (+4.1) with 32x fewer parameters
• Outperforms ColPali (~3B) on multilingual v2 (+7.2) and v3 (+4.5) with single-vector cosine retrieval (no MaxSim)
• Single-vector retrieval — queries and documents share the same 2048-dim embedding space as Qwen3-VL-Embedding-2B; retrieval is a plain dot product, FAISS-compatible, 4 KB per page (float16)
• Lightweight on storage — 282 MB model file; doc index costs 64× less than ColPali's multi-vector patches
• 51 ms CPU query latency — 50x faster than DSE-Qwen2, 143x faster than ColPali
• 6 languages: English, German, French, Spanish, Italian, Portuguese — all >92% teacher retention

Results

NDCG@5 (×100). v1 = 10 English datasets, v2 = 4 multilingual datasets, v3 = 8 multilingual datasets.

Per-Language Retention (v2 + v3, 19,537 queries)

All 6 languages achieve >92% of the 2B teacher's performance.

Quick Start

from sentence_transformers import SentenceTransformer

model = SentenceTransformer("nanovdr/NanoVDR-Q-DistilBERT-Qwen3VL2B-2048-ML")

queries = [
    "What was the revenue growth in Q3 2024?",           # English
    "Quel est le chiffre d'affaires du trimestre?",       # French
    "Wie hoch war das Umsatzwachstum im dritten Quartal?", # German
    "¿Cuál fue el crecimiento de ingresos en el Q3?",     # Spanish
    "Qual foi o crescimento da receita no terceiro trimestre?", # Portuguese
    "Qual è stata la crescita dei ricavi nel terzo trimestre?", # Italian
]
query_embeddings = model.encode(queries)
print(query_embeddings.shape)  # (6, 2048)

# Cosine similarity against pre-indexed document embeddings
# scores = query_embeddings @ doc_embeddings.T

Prerequisites: Document Indexing with Teacher Model

NanoVDR is a query encoder only. Documents must be indexed offline using the teacher VLM (Qwen3-VL-Embedding-2B), which encodes page images into 2048-d embeddings. This is a one-time cost.

# pip install transformers qwen-vl-utils torch
from scripts.qwen3_vl_embedding import Qwen3VLEmbedder

teacher = Qwen3VLEmbedder(model_name_or_path="Qwen/Qwen3-VL-Embedding-2B")

# Encode document page images
documents = [
    {"image": "page_001.png"},
    {"image": "page_002.png"},
    # ... all document pages in your corpus
]
doc_embeddings = teacher.process(documents)  # (N, 2048), L2-normalized

Note: The Qwen3VLEmbedder class and full usage guide (including vLLM/SGLang acceleration) are available at the Qwen3-VL-Embedding-2B model page. Document indexing requires a GPU; once indexed, retrieval uses only CPU.

Full Retrieval Pipeline

import numpy as np
from sentence_transformers import SentenceTransformer

# doc_embeddings: (N, 2048) numpy array from teacher indexing above

# Step 1: Encode text queries with NanoVDR (CPU, ~51ms per query)
student = SentenceTransformer("nanovdr/NanoVDR-Q-DistilBERT-Qwen3VL2B-2048-ML")
query_emb = student.encode("Quel est le chiffre d'affaires?")  # shape: (2048,)

# Step 2: Retrieve via cosine similarity
scores = query_emb @ doc_embeddings.T
top_k_indices = np.argsort(scores)[-5:][::-1]

How It Works

NanoVDR uses asymmetric cross-modal distillation to decouple query and document encoding:

The student is trained to align query embeddings with the teacher's query embeddings via pointwise cosine loss — no document embeddings or hard negatives are needed during training. At inference, student query embeddings are directly compatible with teacher document embeddings.

Training

Multilingual Augmentation Pipeline

1. Extract 489K English queries from the 711K training set
2. Translate each to 5 languages using Helsinki-NLP Opus-MT → 778K translated queries
3. Re-encode translated queries with the frozen teacher in text mode (15 min on H200)
4. Combine: 711K original + 778K translated = 1.49M training pairs
5. Train with halved epochs (10 vs 20) and slightly higher lr (3e-4 vs 2e-4) to match total steps

Efficiency

Model Variants

NanoVDR-Q-DistilBERT-Qwen3VL2B-2048-ML is the recommended model. The other variants are provided for research and ablation purposes.

Want a system with no teacher at all?

Every model in this table targets Qwen3-VL-Embedding-2B at 2048 dimensions, so documents still have to be indexed by that 2B teacher. A second family targets the 8B teacher at 4096 dimensions and includes a document tower, so nothing multi-billion runs at indexing time either:

The two families are not interchangeable: 2048-d and 4096-d vectors live in different spaces and will not score against each other.

Key Properties

Citation

@article{nanovdr2026,
  title={NanoVDR: Distilling a 2B Vision-Language Retriever into a 70M Text-Only Encoder for Visual Document Retrieval},
  author={Liu, Zhuchenyang and Zhang, Yao and Xiao, Yu},
  journal={arXiv preprint arXiv:2603.12824},
  year={2026}
}

License

Apache 2.0