DUSUNEN Pusula 118M v0
A 117.7M-parameter, 384-dimensional Turkish retrieval experiment built to measure the quality, index-memory and browser-deployment trade-off of a smaller encoder. It is a retriever, not a chat or text-generation model.
Five-task measured results
Official MTEB 2.18.16 evaluators, Turkish-only subsets, normalized embeddings and explicit prompt formats were used for every row.
| Model | Params | Dim | TurHist | XQuAD | WebFAQ | MKQA | Belebele | Macro |
|---|---|---|---|---|---|---|---|---|
| multilingual E5 base | 278.0M | 768 | 0.49726 | 0.95335 | 0.65032 | 0.07213 | 0.92503 | 0.619618 |
| DUSUNEN Rota 270M v2 | 268.1M | 640 | 0.42198 | 0.86393 | 0.56886 | 0.10331 | 0.88493 | 0.568602 |
| DUSUNEN Rota 270M v1 | 268.1M | 640 | 0.42196 | 0.85832 | 0.56402 | 0.10296 | 0.88222 | 0.565896 |
| DUSUNEN Pusula 118M v0 | 117.7M | 384 | 0.25299 | 0.81123 | 0.46307 | 0.04855 | 0.82451 | 0.480070 |
Relative to DUSUNEN Rota 270M v1, this model uses 56.1% fewer parameters and 40% smaller float32 vectors, with a 15.2% relative macro-score reduction. The source-domain triplet accuracy rose from 0.8745 to 0.9200. This is a measured size/quality option, not a universal quality replacement.
Use
from sentence_transformers import SentenceTransformer
model = SentenceTransformer("GoktugD/DUSUNEN-Pusula-118M-v0")
query = model.encode("Hard negative neden önemlidir?", normalize_embeddings=True)
docs = model.encode(
["Zor negatifler karar sınırını güçlendirir.", "Ankara Türkiye'nin başkentidir."],
normalize_embeddings=True,
)
print(docs @ query)
Queries and documents are encoded as plain text. Store normalized vectors and use cosine similarity or inner product.
Training and deployment
- Pinned base revision:
e8f8c211226b894fcb81acc59f3b34ba3efd5f42 - Data: 100,000 train / 2,000 validation triplets
- Objective: Cached Multiple Negatives Ranking Loss
- Sequence length: 256; effective batch: 64; BF16; seed: 3407
- Hardware: one RTX 5060 Laptop GPU with 8 GB VRAM
- Training time: 1,397.5 seconds; peak allocated GPU memory: 1.073 GB
The included dynamic-int8 ONNX graph is 118,335,516 bytes. On 16 documents and four probes, PyTorch-to-INT8 mean cosine agreement was 0.990002, minimum agreement was 0.977650 and top-1 agreement was 100%.
Limitations
- The training data is machine-translated Turkish MS MARCO and inherits its translation artifacts, dated facts and web-text biases.
- Compactness reduces retrieval quality on this five-task suite.
- Similarity is a ranking signal, not a calibrated probability or fact check.
- Exact-overlap auditing does not detect paraphrases or semantic duplicates.
The release includes raw MTEB results, training state, environment metadata, ONNX validation and SHA-256 file manifests.