mdbr-leaf-mt-resume-grader
A compact BGE-small embedding model fine-tuned to rank job descriptions for a resume. It powers the opt-in, on-device Semantic AI feature in the NUWorks Co-op Grader Chrome extension.
This is the expanded-label release from 2026-08-27. It replaces the previous OpenAI-label candidate after training on a larger NUWorks label set and passing a fresh unseen-resume bias check.
Evaluation
All numbers below use production-parity mean-vector resume scoring and the
quantized onnx/model_quantized.onnx artifact.
| Held-out dataset | Pairs | Spearman | NDCG@10 | Top-1 |
|---|---|---|---|---|
| Expanded NUWorks test (2 unseen resumes) | 1,944 | 0.8285 | 0.8444 | 0.0% |
| Fresh bias check (24 unseen resumes) | 960 | 0.7684 | 0.8683 | 37.5% |
| Hugging Face external test | 820 | 0.6682 | 0.8627 | 73.9% |
NeuralFrame external-test Spearman is 0.6120. On the fresh bias check, this model beat the previous candidate on all 24 individual resumes; median per-resume Spearman was 0.7956 and 23/24 resumes scored at least 0.50. The two-resume NUWorks Top-1 result is too coarsely sampled to be useful; rank correlation and NDCG are the meaningful metrics for that split.
The fresh bias-check resumes were never used for training or model selection.
After the locked ranking evaluation, their labels were used to fit the
display-only monotonic mapping in calibration.json; it does not change raw
cosine ranking. The jobs came from the existing NUWorks pool, so this result
tests resume generalization rather than a fully new job-description
distribution.
Training
Training used two stages:
- Three CoSENT epochs on 6,804
gpt-5.6-terra/gpt-5.6-luna-labeled NUWorks resume-job pairs, with checkpoint selection on two resume-disjoint validation groups. - One replay epoch on the original external datasets to limit catastrophic forgetting: 6,349 Hugging Face rows weighted 3:1 over 7,630 NeuralFrame rows, producing a 26,677-example replay schedule.
No resume crosses the NUWorks train, validation, or test boundaries. The model uses 384-dimensional normalized embeddings, cosine similarity, and a 256-token window.
Files
model.safetensors: Sentence Transformers / PyTorch weightsonnx/model.onnx: FP32 ONNX exportonnx/model_quantized.onnx: int8 deployment model used by the extensiononnx/model_quint8_avx2.onnx: uint8 AVX2-compatible alternativecalibration.json: revision-pinned cosine-to-display-score mapping
The browser deployment is approximately 34 MB for the quantized model. Resume and job text are processed locally by the extension and are not sent to this repository.
Usage
from sentence_transformers import SentenceTransformer
model = SentenceTransformer("turtlecap/mdbr-leaf-mt-resume-grader")
embeddings = model.encode(
["resume text", "job description"],
normalize_embeddings=True,
)
score = float(embeddings[0] @ embeddings[1])
For extension parity, long resumes should be chunked before embedding and the normalized chunk embeddings averaged and normalized again.