CrosswordSpace++: Italian BERT cross-encoder reranker
The second-stage reranker of CrosswordSpace++. It is a sentence-transformers CrossEncoder that scores (clue, answer) pairs for Italian crosswords.
It reranks the top-100 length-filtered candidates retrieved by the bi-encoder cruciverb-it/crosswordspacepp-dualencoder. The final score is a convex blend of the two models' per-clue softmax-normalized scores, α · CE + (1 − α) · BE with α = 0.17, where α was tuned on the validation set.
Code: https://github.com/snizio/crosswordspacepp
Model Description
- Model type: Cross-encoder (
BertForSequenceClassification, 1 output label, sigmoid activation) - Base model: nickprock/sentence-bert-base-italian-xxl-uncased
- Max sequence length: 84 tokens
- Training data: EVALITA 2026 CruciverbIT Task 1 train split. Each of the 374,766 clues is paired with its gold answer and 9 negatives mined from the bi-encoder candidates (3 hard, 3 medium, 3 easy), for 3,747,628 pairs in total. Gold answers, their synonyms and stem-matches are excluded from the negatives.
- Training objective: binary cross-entropy (
pos_weight = 9), lr 2e-5, batch size 256, 10 epochs, bf16; the best checkpoint by validation reranking MRR@10 is released
Usage
from sentence_transformers import CrossEncoder
model = CrossEncoder("cruciverb-it/crosswordspacepp-reranker")
clue = "Giorni di metà mese nell'antica Roma"
candidates = ["idi", "net", "ier", "die", "vii"]
scores = model.predict([(clue, c) for c in candidates])
print(scores)
# [9.98e-01 9.45e-01 2.05e-03 3.80e-04 2.22e-04]
print(model.rank(clue, candidates))
Evaluation
Results on the EVALITA 2026 CruciverbIT Task 1 test set (20,821 clues). All systems rank the top-100 length-filtered candidates from the bi-encoder.
| System | Acc@1 | Acc@10 | MRR@10 |
|---|---|---|---|
| Bi-encoder only | 57.9 | 80.9 | 65.8 |
| Cross-encoder only (this model) | 59.9 | 83.1 | 67.9 |
| Blend (α = 0.17) | 68.2 | 85.5 | 74.5 |
Citation
TBD
License
CC BY 4.0