ce-cat-distilbert
MiniLM-L12 cross-encoder reranker trained under the configuration stated in §4 of Askari et al., "Injecting the BM25 Score as Text Improves BERT-Based Re-rankers", ECIR 2023: cross-entropy loss with early stopping on validation nDCG@10.
Paired with Amdestya/ce-bm25cat-distilbert — identical data,
schedule and seed; the arms differ only in the input construction.
Input format
text_a = f"{query}"
text_b = passage
# tokenises to: [CLS] query [SEP] passage [SEP]
Matched no-injection baseline. Do NOT inject anything into text_a -- this model has never seen a numeric token there.
Usage
from sentence_transformers import CrossEncoder
model = CrossEncoder("Amdestya/ce-cat-distilbert", max_length=256)
Outputs are raw logits (identity activation), unbounded, higher = more relevant.
Training
| objective | cross-entropy on binary relevance |
| optimizer | Adam, lr 7e-06 |
| batch size | 32 |
| length caps | query 30, passage 200 tokens |
| stopping | early stopping on msmarco-passage/trec-dl-2020/judged nDCG@10, patience 3 |
| steps | 100000 |
| best validation nDCG@10 | 0.7016 |
| seed | 42 |
Built as a reference artifact for a dissertation study on reproducing IR papers with LLMs.