BioMed Reranker
Cross-encoder reranking over biomedical scientific literature (433,449 papers).
BioMed Collection | BioMed Encoder
- ✅ Beats the NLM's own cross-encoder on concept search: handed
ncbi/MedCPT-Cross-Encoder's own candidate pool, R@10 0.544 vs 0.525, p = 0.013. - ✅ +14% top-1 accuracy over the encoder's raw top-50 (R@1 0.269 → 0.306), nDCG@10 +8% — a paired win, significant in both query families.
- ✅ Two-stage system beats every open baseline on concept search, and edges MedCPT's complete retriever + cross-encoder stack (0.549 vs 0.525 R@10).
- ✅ Drop-in: standard
sentence-transformersCrossEncoder, 326 pairs/s on one A10G, Apache-2.0.
A 149M cross-encoder fine-tuned from
Alibaba-NLP/gte-reranker-modernbert-base
to re-score retrieval candidates. Takes (query, document), returns a relevance logit.
Second stage over any dense or hybrid top-K.
Second stage of a two-stage stack — pair it with the BioMed Encoder for best results.
Details
| Property | biomed-reranker |
|---|---|
| Type | Cross-encoder (pointwise reranker) |
| Total parameters | 149M |
| Backbone | Alibaba-NLP/gte-reranker-modernbert-base (ModernBERT) |
| Output | Single relevance logit per (query, document) pair |
| Context | 8,192 tokens native; trained and served at 256 |
| Training signal | 40k mined query→paper groups, hard negatives |
| Built for | Second-stage reranking of search results |
| Pair with | BioMed Encoder (first stage) |
| MeSH concept R@10 | 0.549 (over the encoder's top-50) |
| License | Apache-2.0 |
Performance
MeSH concept queries — expert subject headings, the professional-search case. Benchmark BioMed Eval, frozen before any training, split by paper; 1,241 held-out queries, candidate pools retrieved against all 433,449 papers. Paired exact McNemar and 5,000-sample bootstrap CIs throughout.
What the second stage buys — the same top-50, before and after this model:
| Metric | Encoder top-50 alone | + this reranker | Δ |
|---|---|---|---|
| R@1 | 0.269 | 0.306 | +14% |
| R@10 | 0.524 | 0.549 | +5% |
| nDCG@10 | 0.388 | 0.419 | +8% |
The full two-stage system — this reranker over the BioMed Encoder's top-50, the production configuration:
| Rank | System | R@1 | R@10 | nDCG@10 |
|---|---|---|---|---|
| 1 | BioMed Encoder + this reranker (ours) | 0.306 | 0.549 | 0.419 |
| 2 | ncbi/MedCPT + MedCPT-Cross-Encoder (NLM's full stack) | — | 0.525 | 0.405 |
| 3 | BioMed Encoder, retrieval only | 0.269 | 0.524 | 0.388 |
| 4 | allenai/specter2_base retrieval | 0.153 | 0.351 | 0.247 |
| 5 | Stock BAAI/bge-m3 retrieval | 0.118 | 0.286 | 0.194 |
| 6 | Postgres FTS (keyword search) | 0.047 | 0.081 | 0.063 |
Head-to-head with the NLM's cross-encoder. Handed MedCPT's own candidate pool,
this reranker beats ncbi/MedCPT-Cross-Encoder on concept queries — R@10 0.544 vs
0.525, p = 0.013. MedCPT-Cross-Encoder runs zero-shot on preprints here, while MedCPT's
PubMed training data overlaps substantially with this corpus.
Candidate pool ceiling — R@50 of the retriever that built each pool, the most any reranker can recover:
| Pool | MeSH R@50 |
|---|---|
| BioMed Encoder | 0.673 |
ncbi/MedCPT | 0.624 |
Feed this model a better retriever and its ceiling moves with it.
Training
| Objective | grouped cross-entropy — 1 positive vs hard negatives per group |
| Groups | 40,000 — family mix MESH 61% / AUTHORKW 39%, matching the benchmark |
| Negatives | mined from the retrieval space itself, not sampled at random — a reranker trained on random negatives learns topicality, not ranking |
| Epochs / text budget | 2 / query 600 · document 900 characters (identical to the serving path) |
| Hardware / wall-clock | 1× NVIDIA L40S (g6e.2xlarge) · 42.3 min |
| Run | 20260819-223601-train · 2026-08-20 |
All benchmark gold papers are excluded from training as queries, positives, and negatives.
How to run
from sentence_transformers import CrossEncoder
ce = CrossEncoder("NYSgpt/biomed-reranker", max_length=256)
ce.predict([("gut microbiome depression",
"Interplay of Human Metabolome and Gut Microbiome in Major Depression ...")])
Retrieve a top-50 with the BioMed Encoder, re-score with this model, serve the reordered list. p50 151 ms per query over 50 candidates on one A10G — fast enough to sit behind a live search box on a single cheap GPU.
📬 Contact
Questions, results, or a use case to share? Open a discussion in the Community tab.
Citation
@misc{biomedreranker2026,
title = {BioMed Reranker: a cross-encoder for the complete bioRxiv and medRxiv corpus},
author = {NYSgpt},
year = {2026},
url = {https://huggingface.co/NYSgpt/biomed-reranker}
}