DUSUNEN Pusula 118M v1
A compact Turkish retriever with six selectable vector sizes. Pusula is a 117.7M-parameter Sentence Transformer trained with Matryoshka Representation Learning on 50,000 Turkish hard-negative triplets. It emits 384-dimensional embeddings by default and can be truncated to 256, 192, 128, 64, or 32 dimensions without storing another model.
Measured dimension sweep
All results use the same 2,000-example held-out hard-negative split and cosine similarity. The baseline is the public Pusula v0 checkpoint evaluated with the same code and text format.
| Dimensions | Base accuracy | Pusula v1 accuracy | Gain | Float32 bytes/vector | Index reduction |
|---|---|---|---|---|---|
| 384 | 92.00% | 91.95% | -0.05 pp | 1,536 | 0.0% |
| 256 | 91.60% | 91.70% | +0.10 pp | 1,024 | 33.3% |
| 192 | 90.70% | 90.75% | +0.05 pp | 768 | 50.0% |
| 128 | 90.05% | 89.85% | -0.20 pp | 512 | 66.7% |
| 64 | 88.70% | 88.90% | +0.20 pp | 256 | 83.3% |
| 32 | 86.40% | 86.40% | +0.00 pp | 128 | 91.7% |
These are in-domain held-out triplet results, not a universal or state-of-the-art
claim. Machine-readable evidence is included in dimension-sweep.json.
Turkish retrieval benchmark
The five-task score below was measured with MTEB 2.18.16,
normalized 384-dimensional embeddings, bfloat16 inference, and the plain-text
input format. Dataset revisions and task metadata are recorded in
mteb-five-task.json.
| Task | Main score |
|---|---|
| TurHistQuadRetrieval | 0.25543 |
| XQuADRetrieval | 0.81568 |
| WebFAQRetrieval | 0.46310 |
| MKQARetrieval | 0.04860 |
| BelebeleRetrieval | 0.82598 |
| Macro average | 0.481758 |
Scores across tasks with different metrics should be read as a compact release regression panel, not as a single universal quality measure.
Dynamic-int8 ONNX
The repository includes onnx/model_quantized.onnx (118.3 MB), exported
for AVX2 CPU inference. Across the included validation probes, PyTorch-to-ONNX
mean cosine agreement was 0.989893,
minimum agreement was 0.979842,
and example-query top-1 agreement was
100.0%. See
onnx-validation.json for machine-readable details.
Usage
from sentence_transformers import SentenceTransformer
model = SentenceTransformer(
"GoktugD/DUSUNEN-Pusula-118M-v1",
truncate_dim=128,
)
queries = ["Türkiye'nin başkenti neresidir?"]
passages = [
"Türkiye'nin başkenti Ankara'dır.",
"İstanbul Türkiye'nin en kalabalık şehridir.",
]
q = model.encode(queries, normalize_embeddings=True)
p = model.encode(passages, normalize_embeddings=True)
print(q @ p.T)
Pusula uses plain text; do not add E5-style prefixes. Apply the same
truncate_dim to query and document embeddings.
Training
- Base:
GoktugD/DUSUNEN-Pusula-118M-v0 - Data: 50,000 train / 2,000 held-out Turkish hard-negative triplets
- Loss: cached multiple-negatives ranking loss wrapped by Matryoshka loss
- Dimensions: 384, 256, 192, 128, 64, 32
- Effective batch size: 64; one epoch; bfloat16; seed 3407
- Hardware: NVIDIA GeForce RTX 3090 24 GB
- Training time: 1383.3 seconds
- Peak memory allocated by training: 1.082 GiB
Exact configuration, environment and evaluation scripts are included in this repository.
Limitations
- The dimension sweep is an in-domain held-out evaluation; the separate MTEB panel is broader but is not a substitute for application-specific testing.
- Very small dimensions trade ranking quality for index size.
- The underlying training corpus can contain noisy or semantically overlapping negatives.
- Similarity thresholds require application-specific calibration.
Integrity
model.safetensors SHA-256:
1ac5c7f6c40598ae744b82373eb6f5a37d6569402ebaf5dada12b3e56553d81a
Developed and released by Göktuğ Düşünen.