kenga-embed-prophet-instruct
42M bidirectional UTF-8 byte encoder (seq 160). Instruct prefix matches Giga-Embeddings: query only for retrieval, both sides for STS.
This is not Giga-480M and not a chat model. It does not write replies.
How to encode (same protocol as Giga instruct)
import torch
from modeling_kenga_embed import KengaEmbed
m = KengaEmbed.from_pretrained("GermannM/kenga-embed-prophet-instruct")
# retrieval
q = m.encode_queries(["Where is the capital of Russia?"])
d = m.encode_documents(["Moscow is the capital of Russia."])
print(float((q * d).sum(-1)))
# STS / paraphrase
a = m.encode_sts(["Кот сидит на коврике."])
b = m.encode_sts(["Кошка лежит на ковре."])
Prefixes baked into the loader:
- retrieval query:
Instruct: Given a query, retrieve relevant passages\nQuery: - documents: raw text
- STS:
Instruct: Retrieve semantically similar text.\nQuery:on both sentences
Do not skip the prefix. Scoring z_embed.pt with this template is wrong.
Numbers we measured (2026-09-03)
Handmade holdout, 12 EN + 12 RU triples in embed_holdout.json. Not ruMTEB.
| model | EN | RU | mean |
|---|---|---|---|
| Giga-Embeddings-instruct-480M | 0.833 (10/12) | 0.917 (11/12) | 0.875 |
| this file | 0.833 (10/12) | 1.000 (12/12) | 0.917 |
Prophet misses on that set: Anna Karenina (Leo Strauss trap), H2O vs H2O2.
RuSTSBenchmarkSTS test (1264 pairs, cosine Spearman), CPU, this folder:
0.4416 (was 0.3583 before instruct+STS train). Still far from Giga.
Giga on the same split: 0.8033. A 42M byte encoder is not expected to win STS. Full MTEB(rus, v1) was not run here (pytrec_eval missing in the train env). Please run the official suite yourselves.
Architecture
D=768, L=8, heads=12, dff=3072, V=256 bytes, rank 192/512. Mean-pool + linear. ~42M params. Checkpoint acc_en=0.833 acc_ru=1.0, step 10400.
License
MIT. Train code lives in the kenga-lang / z-system tree, not in this repo.