Noql: A Tiny and Efficient Persian Text Embedding Model
Noql is a Persian sentence embedding model with only 12M parameters. It is trained with Matryoshka representation learning, so its 768-dimensional embeddings can be truncated to 512, 256, 128 or 64 dimensions with little loss in quality.
| Parameters | 11.9M |
| Architecture | ALBERT-base, mean pooling, L2-normalized |
| Embedding size | 768 (Matryoshka: 512 / 256 / 128 / 64) |
| Max sequence length | 512 tokens |
| Similarity | cosine |
Training
Noql was trained on the Shiraz dataset in two stages:
- Positive pairs: query–document pairs with in-batch negatives.
- Hard negatives: half positive pairs, half pairs with mined hard negatives.
Each stage used 2M random samples.
Usage
pip install -U sentence-transformers
from sentence_transformers import SentenceTransformer
model = SentenceTransformer("shekar-ai/Noql")
sentences = [
"پایتخت ایران تهران است.",
"تهران بزرگترین شهر ایران است.",
"امروز هوا بارانی است.",
]
embeddings = model.encode(sentences)
print(model.similarity(embeddings, embeddings))
# tensor([[1.0000, 0.8143, 0.3462],
# [0.8143, 1.0000, 0.2109],
# [0.3462, 0.2109, 1.0000]])
Retrieval
query = "بهترین زمان برای سفر به شیراز چه موقع است؟"
documents = [
"بهار به دلیل هوای معتدل و شکوفههای نارنج بهترین فصل سفر به شیراز است.",
"قیمت طلا در بازار امروز کاهش یافت.",
"حافظیه و سعدیه از جاذبههای معروف شیراز هستند.",
]
scores = model.similarity(model.encode(query), model.encode(documents))
print(scores)
# tensor([[0.8091, 0.0271, 0.5199]])
Smaller embeddings
model = SentenceTransformer("shekar-ai/Noql", truncate_dim=256)
embeddings = model.encode(documents) # shape: (3, 256)
Evaluation
Results on FaMTEB (MTEB(fas, v2), all 52 tasks) at each embedding size.
| Metric | 768 | 512 | 256 | 128 | 64 |
|---|---|---|---|---|---|
| Mean (task) | 58.40 | 58.38 | 58.02 | 57.79 | 56.34 |
| Mean (type) | 62.40 | 62.42 | 62.07 | 62.03 | 60.76 |
| % of 768 score | 100% | 100.0% | 99.4% | 99.0% | 96.5% |
| Storage per vector (fp32) | 3,072 B | 2,048 B | 1,024 B | 512 B | 256 B |
| Task type | Tasks | 768 | 512 | 256 | 128 | 64 |
|---|---|---|---|---|---|---|
| Retrieval | 17 | 47.90 | 47.74 | 47.54 | 46.99 | 44.90 |
| Reranking | 2 | 65.25 | 64.98 | 65.31 | 65.32 | 64.56 |
| Pair Classification | 7 | 79.68 | 79.75 | 79.72 | 79.71 | 79.53 |
| Classification | 16 | 58.06 | 57.89 | 57.45 | 57.00 | 55.79 |
| Clustering | 5 | 59.25 | 60.40 | 59.24 | 60.35 | 58.24 |
| STS | 2 | 70.87 | 70.84 | 70.82 | 70.74 | 70.45 |
| Bitext Mining | 3 | 55.78 | 55.32 | 54.40 | 54.10 | 51.85 |