OneBrain Rerank v1
The first entry in the OneBrain model line — the cross-encoder reranker used by OneBrain native search (Tier-2 precision stage, CLI v3.4.7+).
v1 is not fine-tuned. It is a size/speed-optimized int8 build of BAAI/bge-reranker-v2-m3 (Apache-2.0). Future versions (v2+) may be fine-tuned for OneBrain's vault-search domain; each version ships as its own repo and is sha256-pinned by the OneBrain CLI.
Intended use
- For: reranking a small set of
(query, passage)candidates returned by a first-stage retriever (BM25 / vector), producing a calibrated 0–1 relevance score. This is the Tier-2 precision stage of OneBrain native search. - Not for: retrieval or embedding — it scores query–passage pairs, it does not produce vectors. It is a general (not domain-fine-tuned) reranker, so it is not tuned for any specialized corpus in v1.
Provenance
- Base:
BAAI/bge-reranker-v2-m3(Apache-2.0) - ONNX export:
rozgo/bge-reranker-v2-m3(fp32, 2.27 GB) - Quantization: dynamic int8 (QInt8 weights), onnxruntime 1.27.0, single-file output (
DefaultTensorType=FLOATfor >2GB shape-inference skip) model_int8.onnx— 569,011,484 bytes · sha256dd7b26f4a233732aefbe857bef026050582dc7c1bdb8aeda909080bf15b2ad88
Why int8?
OneBrain runs local-first and CPU-only (no GPU assumption; Raspberry-Pi-class hardware is the floor). A cross-encoder scores every candidate at query time, so both download size and CPU latency matter:
- Size — the fp32 ONNX export is 2.27 GB; this int8 build is 569 MB (~4× smaller). That is the difference between a reranker most users won't download and one that can ship on by default.
- CPU latency — dynamic-QInt8 integer kernels run faster on CPU. OneBrain's design target is ≤500 ms P50 for the Tier-2 stage on a warm daemon at the default candidate depth (a design target, not a published benchmark — no measured latency number is claimed here).
- Quality cost — negligible — quantization perturbs the raw logits slightly, but reranking only needs relative ordering and calibrated separation, both preserved. On the validation pairs below the sigmoid-score delta vs fp32 is ≤ 0.0046 and the pairwise ranking is identical.
int8 is not more accurate than fp32 — it is the same weights at ~¼ the size and lower CPU cost, with quality loss small enough to be irrelevant for ranking. That trade is what makes cross-encoder reranking viable on the hardware OneBrain targets.
Validation (int8 vs fp32, sigmoid(logit))
| pair | fp32 | int8 | delta |
|---|---|---|---|
| "what is panda?" / panda passage | 0.9541 | 0.9495 | 0.0046 |
| "what is panda?" / unrelated (Eiffel Tower) | 0.0000 | 0.0000 | 0.0000 |
| "how to install rust compiler" / rustup passage | 0.9848 | 0.9866 | 0.0018 |
| "วิธีตั้งค่า daemon" / relevant Thai passage | 0.9979 | 0.9978 | 0.0002 |
| "วิธีตั้งค่า daemon" / unrelated Thai passage | 0.0000 | 0.0000 | 0.0000 |
Pairwise ranking identical to fp32; max delta 0.0046.
Calibration (OneBrain vault golden set)
Measured with the int8 model on OneBrain's internal golden set — a ~585-note personal Obsidian vault (Thai/English mixed), 20 answerable + 10 known-no-answer queries. This is an internal evaluation on one real vault, not a public benchmark (MTEB/BEIR/MIRACL were not run).
| bucket | top-hit sigmoid score |
|---|---|
| genuine relevant match | 0.73 – 0.99 |
| tangential / weak match | 0.20 – 0.52 |
| genuine no-answer query | 0.003 – 0.066 (median 0.011) |
The cross-encoder separates real matches from no-answer queries by ~an order of magnitude — the property a bi-encoder's overlapping cosine scores cannot provide, and the reason Tier-2 exists. OneBrain uses these measured bands: > 0.60 confident, 0.30 – 0.60 possible, < 0.30 no strong match (default gate 0.30).
Usage
Consumed automatically by the OneBrain CLI (onebrain search) — the model downloads on first reindex and is verified against the pinned sha256. Direct use: tokenize (query, passage) with the included tokenizer (max_length 512), feed input_ids + attention_mask (int64) → logits [batch, 1] (raw logit; apply sigmoid for a 0–1 relevance score). Multilingual (Thai/English first-class) is inherited from the bge-reranker-v2-m3 base.
Limitations
- Not fine-tuned (v1): a size/speed-optimized build of a general reranker, not adapted to any specific domain.
- Evaluation scope: validated on OneBrain's internal vault golden set, not on public reranking benchmarks — no MTEB/BEIR/MIRACL scores are claimed.
- 512-token input: longer passages are truncated; OneBrain chunks documents upstream so the reranker sees chunk-sized passages.
- int8 perturbation: quantization shifts logits slightly vs fp32 (≤ 0.0046 on the validation pairs) — negligible for ranking, but present.
- CPU-oriented: optimized for CPU int8 inference; users who need maximum fidelity on GPU may prefer the fp32 base model.
License
Apache-2.0, matching the base model.