nllb-rus-tyv-v3-dict_16.5k
Russian ↔ Tyvan (Tuvan) translation model. Fine-tuned from facebook/nllb-200-distilled-600M with an aggressively pruned vocabulary: from ~256k to 16.5k subword pieces (keeping only Russian/Tyvan tokens seen in the corpus, plus Latin letters, digits, punctuation, and special tokens). This cuts the model from 615M to 370M parameters (−40%) while improving quality over the full-vocabulary baseline.
Tyvan language code: tyv_Cyrl (added; embedding initialized from Kyrgyz kir_Cyrl).
Russian: rus_Cyrl.
Results (test, chrF++ / BLEU, greedy)
Evaluated on a held-out test set of 2,000 pairs (seed=42), identical for both systems.
| System | Vocab | Params | ru→tyv chrF++ | tyv→ru chrF++ | ru→tyv BLEU | tyv→ru BLEU |
|---|---|---|---|---|---|---|
| Full-vocab baseline | 256k | 615M | 48.12 | 47.50 | 19.00 | 23.01 |
| This model (pruned) | 16.5k | 370M | 48.50 | 48.11 | 19.79 | 23.20 |
The pruned model is smaller and slightly more accurate on all four metrics. chrF++ is the primary metric (character-level, more reliable than BLEU for agglutinative Turkic languages).
Usage
from transformers import AutoModelForSeq2SeqLM, AutoTokenizer
model_id = "Agisight/nllb-rus-tyv-v3-dict_16.5k"
tok = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForSeq2SeqLM.from_pretrained(model_id)
def translate(text, src, tgt):
tok.src_lang = src
inputs = tok(text, return_tensors="pt")
out = model.generate(**inputs, forced_bos_token_id=tok.convert_tokens_to_ids(tgt),
max_new_tokens=128, num_beams=1)
return tok.batch_decode(out, skip_special_tokens=True)[0]
print(translate("Привет, как дела?", "rus_Cyrl", "tyv_Cyrl"))
print(translate("Экии!", "tyv_Cyrl", "rus_Cyrl"))
Training
- Base: facebook/nllb-200-distilled-600M
- Data: Agisight/tyv-rus-200k (~296k pairs)
- Vocabulary pruning: SentencePiece pieces restricted to those appearing in the corpus
plus ASCII letters/digits/punctuation; embeddings and final logits transferred to the
pruned vocab;
tyv_Cyrladded and initialized fromkir_Cyrl. - Objective: standard seq2seq cross-entropy, both directions, Adafactor (lr 1e-4), batch 48, max_len 96, greedy eval, early-stopping on dev chrF++ plateau.
- Best checkpoint: dev avg chrF++ 48.45 (step 40,000).
Intended use & limitations
- Designed for lightweight / on-device ru↔tyv translation (web, CPU, Apple Silicon).
- Out-of-vocabulary characters (other scripts, emoji) fall back to byte tokens or
<unk>— acceptable for the ru/tyv/Latin scenario it targets. - Trained on short sentences; very long inputs are truncated at 96 tokens during training.
- Not a general multilingual model — only Russian ↔ Tyvan.
Citation
If you use this model, please cite the dataset and base model (NLLB-200, Meta AI).