RetrievalRouter (λ=0.5)

Official checkpoint from RetrievalRouter: Joint Modality and Architecture Selection for Document Retrieval (EMNLP 2026). Given only the query text, RetrievalRouter predicts which retrieval pipeline — across modality (text vs. multimodal) and architecture (lexical, dense, or late-interaction rerank) — to run for that query.

• 📄 Paper: https://arxiv.org/pdf/2608.25625
• 💻 Code: https://github.com/emrekuruu/retrieval-router
• 🤗 Collection (all checkpoints + datasets): https://huggingface.co/collections/emrekuruu/retrieval-router

Motivation

Retrieval pipelines differ in modality (search over text, or over page images) and architecture (cheap dense search, or expensive late-interaction). The accurate ones are slow; the fast ones miss evidence on hard documents. And which one fails depends on the query — a text pipeline can't answer "what's the red curve in Figure 3?", but a multimodal one is overkill for a plain factoid. Across 11 benchmarks, no single pipeline wins on everything. RetrievalRouter picks the cheapest pipeline that can still answer each query, so easy queries stay fast and hard ones still get the heavy pipeline.

What this model is for

This is a router, not a retriever. It takes a query and predicts which of five retrieval pipelines to run — in about 15 ms, before any search happens. You then run the chosen pipeline to fetch documents.

Use it when you keep several retrieval setups over the same corpus and want to run the expensive ones only when they help. Pick the checkpoint by λ: 0.0 for best accuracy, 1.0 for best speed, in between to trade off.

It doesn't rank or read documents itself, and assumes your indices already exist. Trained on English financial, scientific, and open-domain documents; other domains and languages are untested.

This checkpoint

Trained with λ=0.5 — a balanced objective (λ=0.5), trading a controlled amount of accuracy for lower latency between the quality-only (λ=0.0) and latency-only (λ=1.0) endpoints.

Routing arms

The action space is these five arms. Two further pipelines evaluated in the paper (Text-Late, MM-Late) are reported as static reference baselines but never routed to.

Architecture

• Encoder: Qwen/Qwen3-0.6B-Base with LoRA adapters on the attention and feed-forward projections (merged into these weights).
• Pooling: mean-pool over the final hidden states → a 1024-d query representation.
• Head: a single linear layer → logits over the five arms; softmax gives the routing policy.
• Custom modeling code ships in the repo and loads via trust_remote_code=True.

Usage

import torch
from transformers import AutoModel, AutoTokenizer

repo = "emrekuruu/RetrievalRouter-lambda-l50"
tokenizer = AutoTokenizer.from_pretrained(repo, trust_remote_code=True)
model = AutoModel.from_pretrained(repo, trust_remote_code=True).eval()

inputs = tokenizer("In figure 3, what does the red dashed curve represent?",
                   return_tensors="pt", truncation=True, max_length=128)
with torch.no_grad():
    logits = model(**inputs)["logits"]              # shape [1, 5]
arm = model.config.strategy_names[logits.softmax(-1).argmax(-1).item()]
print(arm)   # e.g. "MULTIMODAL_RERANK" -> run that pipeline for this query

The router returns which retrieval pipeline to run, not documents. You then execute the selected pipeline against your own indices.

Training

Trained against soft targets from a per-query reward vector over the five arms, rather than a single hard best-pipeline label (pipelines frequently tie on nDCG@5, and hard labels inject noise). The reward combines accuracy and efficiency,

$$ r_i(q) = (1-\lambda), s_i(q) + \lambda,(1 - \ell_i(q)), $$

where $s_i(q)$ is the arm's nDCG@5 and $\ell_i(q)$ its per-query normalized latency. The reward vector becomes a target distribution via a low-temperature softmax (τ=0.1), and the router minimizes the KL divergence to it. λ is the only knob that differs across the checkpoints above. Training data spans 85,103 queries across 11 benchmarks.

Results (headline)

Against the strongest static pipeline, RetrievalRouter is +2.5% nDCG@5 and 12.4× faster. Against the prior adaptive strategy-selection baseline (emrekuruu/RetrievalRouter_Baseline), it achieves significantly higher nDCG@5 in accuracy-oriented settings and matches or numerically beats it on both nDCG@5 and latency in latency-oriented settings. See the paper for full tables and significance tests.

Citation

@misc{kuru2026retrievalrouterjointmodalityarchitecture,
      title={RetrievalRouter: Joint Modality and Architecture Selection for Document Retrieval}, 
      author={Emre Kuru and Mehmet Onur Keskin and Reza Farahbakhsh and Noel Crespi},
      year={2026},
      eprint={2608.25625},
      archivePrefix={arXiv},
      primaryClass={cs.IR},
      url={https://arxiv.org/abs/2608.25625}, 
}