tevatron3-reranker-8b-contrastive
A pointwise passage reranker built on Qwen3-8B (dense), trained with the Tevatron 3.0 toolkit.
- Backbone: Qwen3-8B (dense)
- Training backend: Megatron
- Objective: contrastive
- Parameter efficiency: full fine-tuning
- Training data: RLHN-680K
Scoring contract (important)
This checkpoint is a causal-LM yes/no scorer, saved as Qwen3ForCausalLM
(Qwen3MoeForCausalLM for MoE). The relevance score of a (query, passage) pair
is the yes/no log-odds at the final prompt token:
score = logit(" yes") − logit(" no")
evaluated right after the prompt suffix "\nIs the passage relevant to the query?".
The prompt is "query: {{query}} passage: {{title}} {{text}}\nIs the passage relevant to the query?"
and no EOS token is appended.
This differs from the HF sequence-classification rerankers in this release (
*-hf,*-lora-hf), which score a single regression logit at EOS.
Usage
Score with Tevatron's reranker eval backend (see the Tevatron repo), or directly:
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
name = "brutusxu/tevatron3-reranker-8b-contrastive"
tok = AutoTokenizer.from_pretrained(name)
model = AutoModelForCausalLM.from_pretrained(name, dtype=torch.bfloat16).cuda().eval()
yes_id = tok.encode(" yes", add_special_tokens=False)[-1]
no_id = tok.encode(" no", add_special_tokens=False)[-1]
prompt = ("query: what is the capital of france "
"passage: Paris is the capital of France.\n"
"Is the passage relevant to the query?")
ids = tok(prompt, return_tensors="pt").to("cuda")
with torch.no_grad():
logits = model(**ids).logits[0, -1]
print((logits[yes_id] - logits[no_id]).item())
Citation
If you use this model, please cite the Tevatron toolkit.