ce-bm25cat-distilbert
MiniLM-L12 cross-encoder reranker trained under the configuration stated in §4 of Askari et al., "Injecting the BM25 Score as Text Improves BERT-Based Re-rankers", ECIR 2023: cross-entropy loss with early stopping on validation nDCG@10.
Paired with Amdestya/ce-cat-distilbert — identical data,
schedule and seed; the arms differ only in the input construction.
Input format
text_a = f"{query} [SEP] {bm25_score}"
text_b = passage
# tokenises to: [CLS] query [SEP] score [SEP] passage [SEP]
Paper Eq. 3 ordering: the score sits BETWEEN query and passage. This differs from Amdestya/bm25cat-minilm-l12, which follows the authors' released code and puts the score BEFORE the query. The two are not interchangeable.
Normalise the score the same way
bm25_score = int((raw_bm25 / 50) * 100) # global Min-Max, fixed constants, then x100 as an integer
Min-Max in the global setting (fixed min=0, max=50, not per-query), scaled by 100 and truncated
to an integer — paper §3.3 row (f). Training scores came from Anserini/Lucene BM25 with
k1=0.82, b=0.68. Other BM25 implementations produce a different score distribution.
Usage
from sentence_transformers import CrossEncoder
model = CrossEncoder("Amdestya/ce-bm25cat-distilbert", max_length=256)
Outputs are raw logits (identity activation), unbounded, higher = more relevant.
Training
| objective | cross-entropy on binary relevance |
| optimizer | Adam, lr 7e-06 |
| batch size | 32 |
| length caps | query 30, passage 200 tokens |
| stopping | early stopping on msmarco-passage/trec-dl-2020/judged nDCG@10, patience 3 |
| steps | 120000 |
| best validation nDCG@10 | 0.7104 |
| seed | 42 |
Built as a reference artifact for a dissertation study on reproducing IR papers with LLMs.