reranker-bert-italian-uncased-mmarco-mnrl

Why this model

Italian-language information retrieval has very few dedicated reranker models — most pipelines either fall back to multilingual cross-encoders (diluted by 100 languages) or skip reranking entirely. This model is trained natively in Italian, on Italian-specific hard negatives, to fill that gap.

Results

Evaluated with CrossEncoderRerankingEvaluator on a held-out 2,500-sample dev split of the training distribution:

"Base retriever" refers to nickprock/multi-sentence-BERTino — a strong, Italian-specialized bi-encoder — used to mine hard negatives during training. The gain reported here is therefore a realistic estimate of what this reranker adds on top of a competitive existing Italian retriever, not a comparison against a weak baseline.

Training data

The model was trained on a high-quality Italian passage ranking mixture combining samples from mMARCO-it and vincolle/gooaq-italian-80k, an Italian translation of GooAQ that I produced myself (translated with DeepSeek). To my knowledge this is the first public Italian version of GooAQ — it does not exist as a native dataset in any other language besides English.

Hard negative mining

Hard negatives were mined using nickprock/multi-sentence-BERTino, a native Italian bi-encoder trained with Matryoshka Representation Learning. Using an Italian-specific embedder for mining (rather than a generic multilingual one) was a deliberate choice — early experiments with multilingual embedders produced negatives that were either too easy (trivially distinguishable) or, in the worst case, contained answers just as valid as the labeled positive. Switching to an Italian-native retriever for mining produced negatives that were genuinely hard without being false negatives, which is what made margin-based filtering during mining meaningful at all.

• 75,000 training triples (question, positive, 5 hard negatives)
• 2,500 held-out dev samples, reranking-evaluation format (30 candidates per query, positive included)

Usage

pip install -U sentence-transformers

from sentence_transformers import CrossEncoder

model = CrossEncoder("vincolle/reranker-bert-italian-uncased-mmarco-mnrl")

# Score query-document pairs directly
pairs = [
    ("Quante calorie si bruciano correndo 5 miglia?",
     "Correndo 5 miglia si brucia una grande quantità di calorie. In generale, la maggior parte delle persone che corre 5 miglia brucia circa 500 calorie."),
    ("Quante calorie si bruciano correndo 5 miglia?",
     "Le pinzette sono consentite nel bagaglio a mano, così come i rasoi elettrici."),
]
scores = model.predict(pairs)
print(scores)
# [0.99..., 0.05...]

# Or rerank a list of candidate documents for a single query
ranks = model.rank(
    "Quante calorie si bruciano correndo 5 miglia?",
    [
        "Correndo 5 miglia si brucia una grande quantità di calorie...",
        "Le pinzette sono consentite nel bagaglio a mano...",
        "Guidare con ammortizzatori difettosi può essere estremamente pericoloso...",
    ],
)

Suggested pipeline

This reranker is designed as the second stage of a retrieve-then-rerank pipeline:

1. Retrieve top-k candidates with a fast bi-encoder (e.g. nickprock/multi-sentence-BERTino)
2. Rerank those candidates with this model to get the final ranking

from sentence_transformers import SentenceTransformer, CrossEncoder

retriever = SentenceTransformer("nickprock/multi-sentence-BERTino")
reranker = CrossEncoder("vincolle/reranker-bert-italian-uncased-mmarco-mnrl")

query = "Quante calorie si bruciano correndo 5 miglia?"
candidates = retriever.similarity(...)  # your top-k retrieval step
reranked = reranker.rank(query, candidates)

Training details

Training curve

Framework versions

• Python 3.12.13 · Sentence Transformers 5.5.1 · Transformers 5.10.2 · PyTorch 2.11.0+cu128 · Datasets 4.0.0

Citation

If you use this model, please cite the base model and the Sentence Transformers library:

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}