zerank-2-reranker — MLX (bf16)

zeroentropy/zerank-2-reranker converted to MLX for Apple silicon. This is the unquantized bfloat16 conversion — every weight is stored at the same precision as the original checkpoint, so it is the highest-fidelity build and the reference the quantized ones are measured against.

On-disk size: 8.1 GB.

zerank-2 is a cross-encoder reranker built on Qwen3-4B. It reads a query and one document together and returns a single relevance score, which is more accurate than comparing independent embeddings but requires one forward pass per candidate document.

Usage

Download rerank.py from this repo, then:

pip install mlx-lm transformers

from rerank import Reranker, probability

reranker = Reranker("fcmeyer/zerank-2-reranker-mlx-bf16")

ranked = reranker.rank(
    "What causes the northern lights?",
    [
        "Auroras occur when charged particles from the solar wind are funnelled "
        "along Earth's magnetic field lines into the upper atmosphere.",
        "Iceland is a popular destination for tourists hoping to photograph them.",
        "The Earth's magnetic field reverses polarity every few hundred thousand years.",
    ],
)

for document, score in ranked:
    print(f"{score:+.3f}  p={probability(score):.3f}  {document[:60]}")

rank() returns documents best-first. score() takes explicit (query, document) pairs and preserves your input order. Both batch internally.

How scoring works

The model does not have a classification head. A pair is rendered through the chat template as system = query, user = document, followed by an assistant generation prompt; the score is the logit of token id 9454 ("Yes") at the final position. That token id comes from 1_LogitScore/config.json in this repo.

The raw logit is the score, and it is what you rank by. sigmoid(score / 5) maps it to the 0–1 range shown on the upstream model card. Scores are comparable within a single query, not across queries.

Verification

Every score below is a "Yes" logit measured on the same 486 query/document pairs — graded relevance across ten domains, five languages, code, deliberately misleading lexical overlap, empty and single-character documents, and a length ladder running past the 32k truncation boundary.

The reference is PyTorch float32. An absolute tolerance would be the wrong yardstick here: bfloat16 carries about three significant digits, and a 36-layer forward pass accumulates rounding differently on different kernels. So PyTorch's own bfloat16 was measured on the same machine and same pairs, and serves as the noise floor — the question is not whether MLX differs from float32, but whether it differs by more than bfloat16 already does.

MLX bfloat16 is closer to float32 than PyTorch's own bfloat16 is on worst-case error (0.2172 vs 0.3274), and within a few thousandths of it on every other metric above — neither implementation reorders a single distinguishable pair (see below).

Per-query Kendall τ is computed over each query's own candidate documents — the ordering a user actually sees. It sits near 0.98 rather than 1.0 for both implementations because most candidates within a query are separated by only a few tenths of a logit, and bfloat16 cannot resolve gaps that small. The row that matters is the next one: an inversion is counted as resolvable only when the two documents' float32 scores differ by more than 0.1930, the 99th-percentile error PyTorch's bfloat16 itself makes. By that measure neither implementation reorders a single distinguishable pair. A "verdict flip" is a pair crossing the sigmoid(score/5) = 0.5 boundary relative to float32, again only counted once float32 itself resolved the pair.

Independent verification

The pairs above are hand-authored for coverage, not sampled independently of this repo. As a second check, the same three-way comparison was repeated on 1449 query/document pairs drawn from public BEIR reranking benchmarks (FiQA, SciFact, NFCorpus — finance, STEM, and medical, matching the domains above) that this repo did not write.

Same gates, same result: MLX matches or beats PyTorch's own bfloat16 noise floor on every metric.

Differences from the source repo

Only the weight format. This repo carries the full upstream file set, including 1_LogitScore/config.json, modules.json, sentence_bert_config.json, config_sentence_transformers.json, and the BPE vocab.json / merges.txt — mlx_lm.convert copies none of those on its own, and without 1_LogitScore/config.json there is no record in the repo of which token id the score comes from.

Reproducing

pip install mlx-lm
mlx_lm.convert --hf-path zeroentropy/zerank-2-reranker \
    --mlx-path zerank-2-mlx-bf16 --dtype bfloat16

then copy the sentence-transformers metadata listed above across from the source repo.

License and attribution

Apache 2.0, inherited from the source model. All credit for the model itself goes to ZeroEntropy; see their technical report. This repo is a format conversion and adds no training.