Vela Reranker
Bring relevant context to the top. Vela Reranker scores passages for retrieval, RAG, and context selection in your router.
307M encoder · 32K context · Multilingual
Evaluation
Compared with the original mmBERT Reranker. Scores are ×100; higher is better.
| Evaluation | Metric | Original mmBERT | Vela |
|---|---|---|---|
| MIRACL · 320 queries, 4 languages | nDCG@10 | 80.47 | 87.11 |
| Natural Questions · 128 queries | nDCG@10 | 97.86 | 99.13 |
| SciFact · 48 queries, all 5,183 documents | nDCG@10 | 85.51 | 91.21 |
| QASPER · 64 queries | nDCG@10 | 56.73 | 62.86 |
| Controlled 4K–32K context · 288 pairs | Pair accuracy | 55.21 | 62.50 |
| 32K context subset · 72 pairs | Pair accuracy | 56.94 | 56.94 |
| 32K end-position subset · 24 pairs | Pair accuracy | 54.17 | 54.17 |
Matched development subsets with identical candidate pools and complete inputs, using 22 layers, 768 dimensions, and FP32. MIRACL covers Arabic, Spanish, Japanese, and Chinese. The context set tests 24 queries across four lengths and three positions; its views are not independent examples.
Selected instruction-retrieval tasks from MTEB 2.20.12:
| Task | Metric | Original mmBERT | Vela |
|---|---|---|---|
| Core17InstructionRetrieval | p-MRR | 1.92 | 2.37 |
| News21InstructionRetrieval | p-MRR | 6.56 | 3.23 |
| Robust04InstructionRetrieval | p-MRR | -1.06 | 2.25 |
These results describe the listed tasks and development subsets, not a full MTEB score or ranking.
Quick start
Install torch, transformers, and safetensors. Use a ROCm-enabled PyTorch build for AMD GPUs.
import torch
from transformers import AutoModel, AutoTokenizer
model_id = "llm-semantic-router/Vela-1.0-Encoder-307M-Reranker"
device = "cuda" if torch.cuda.is_available() else "cpu"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModel.from_pretrained(model_id, trust_remote_code=True).to(device).eval()
query = "How do I reset my password?"
passages = [
"Select Forgot password on the sign-in page to receive a reset link.",
"Your monthly invoice is available in the billing dashboard.",
]
inputs = tokenizer(
[query] * len(passages), passages,
padding=True, truncation=False, return_tensors="pt",
).to(device)
with torch.inference_mode():
scores = model(**inputs).logits.flatten().tolist()
print(sorted(zip(scores, passages), reverse=True))
Higher scores mean greater relevance. The default uses all 22 layers and 768 dimensions. The 32,768-token limit includes the query, passage, and special tokens.