SKILLRET-Edge-109M
A 109.5M-parameter bi-encoder for agent skill retrieval — picking the right skill out of a
catalogue for a natural-language request. Distilled from ThakiCloud/SKILLRET-Embedding-0.6B
and small enough to run on a CPU next to the agent.
Scored on the public ThakiCloud/SKILLRET test split (4,392 queries / 6,006 skills), the metric the SkillRet paper uses as its headline.
NDCG@10 79.18 · 219 MB fp16
Correction — September 2026. Earlier versions of this card reported the SKILLRET-Embedding-0.6B teacher as 80.82 NDCG@10 on the current 4,392-query / 6,006-skill evaluation split. That score was measured before the train/evaluation query-prefix contract was corrected. Re-evaluation on
ThakiCloud/SKILLRETrevisiona050ad2under the canonical query contract gives 78.48 NDCG@10. The model weights are unchanged. The earlier 605 MB size was also not the serialized artifact size; the distributed BF16 checkpoint is 1191.6 MB (decimal MB). We retain this note so results quoted from earlier versions of the card can be interpreted correctly.
Results
| Variant | Size on disk | NDCG@10 | vs teacher |
|---|---|---|---|
| SKILLRET-Embedding-0.6B (teacher) | 1191.6 MB | 78.48 | — |
| fp16 | 219.0 MB | 79.18 ± 0.42 | 98.0% |
| int8 / g16 | 136.9 MB | 79.18 ± 0.42 | 98.0% |
| int4 / g16 | 82.0 MB | 79.21 ± 0.42 | 98.0% |
| int3 / g16 | 68.4 MB | 78.04 ± 0.44 | 96.6% |
This model recovers 98% of a 26x larger teacher. int8 and int4 are free; int3 costs 1.14pp and is the point where quantization starts to be a real trade.
Usage
from sentence_transformers import SentenceTransformer
m = SentenceTransformer("ThakiCloud/SKILLRET-Edge-109M")
# ⛔ Encode queries BARE — no instruction prefix. See query_prefix.json.
q = m.encode(["I need to convert a spreadsheet into a chart"], normalize_embeddings=True)
d = m.encode(["Chart Builder — turn tabular data into bar/line charts"], normalize_embeddings=True)
print(q @ d.T)
How it was trained
Knowledge distillation from ThakiCloud/SKILLRET-Embedding-0.6B (NDCG@10 78.48 on the
same split), kd_weight=0.7, multi-positive InfoNCE over 1–3 golds per query, 12 epochs,
cosine schedule. The epoch was chosen on a skill-disjoint holdout — the test split was
never used for selection.
⚠️ Read this before you compare numbers
Use the same query prefix at train and eval time. This repo ships
query_prefix.json recording the contract (resolved: "", i.e. queries are encoded
bare, with no instruction prefix). Which prefix you pick barely matters after
fine-tuning — none 79.18 vs an instruction prefix 78.50, inside the standard error —
but keeping it consistent matters a great deal. We measured a +8.84pp swing on a
single checkpoint purely from a train/eval prefix mismatch, and that mismatch also
manufactured a fake result in which quantization appeared to beat fp16. It does not.
Other caveats worth stating plainly:
- The model card of the SkillRet reference models reports a 4,997-query / 6,660-skill split. That split is not in the currently published dataset — we verified the public files are hash-identical to ours at 4,392 / 6,006. Do not convert between the two.
- Standard error on this split is about ±0.45. Differences under ~1pp are not rankings.
- Pooling is CLS, embeddings are L2-normalized,
max_length=256at evaluation.
What did NOT work (so you don't repeat it)
| Attempt | Result |
|---|---|
| Distilling from the 8B teacher instead of 0.6B | −2.40pp (capacity gap) |
| INT2 / ternary post-training quantization | Total collapse (~0.1 NDCG@10) — GPTQ and QuIP do not rescue it |
| Hard-negative mining (our mined set) | −1.0pp |
| LEAF auxiliary projection loss (w=0.3) | −1.51pp (paired t = −7.78) |
License
Apache-2.0, inherited from the base model.
Citation
Benchmark: SkillRet, arXiv:2605.05726
Paper (measurements behind the quantized variants and the student frontier): Where Post-Training Quantization Breaks Text Embedders: A Measured Map Across Four Embedder Families, arXiv:2609.16391
@article{han2026ptqembedders,
title = {Where Post-Training Quantization Breaks Text Embedders: A Measured Map Across Four Embedder Families},
author = {Han, Hyojung},
journal = {arXiv preprint arXiv:2609.16391},
year = {2026}
}