Russian ↔ Kabardian bidirectional translator (MarianMT, 61M) — version 2
One small model translates in both directions between Russian and Kabardian (East Circassian, kbd). The
direction is chosen by a target tag at the start of the source text: >>kbd<< (into Kabardian) or >>ru<< (into
Russian).
Version 2 (this revision) is a new training from Helsinki-NLP/opus-mt-zle-zle
on our cleaned corpus v2 and on re-translated synthetic data, with the same tokenizer and the same input/output
format as version 1. On FLORES-200 it reaches chrF 59.6 for ru→kbd and 51.3 for kbd→ru (version 1: 55.9 / 49.6).
Version 1 stays available at the revision v1.
| architecture | MarianMT, 6 + 6 layers, d_model 512, 8 heads, 61M parameters |
| tokenizer | SentencePiece unigram, Russian + Kabardian, 32 045 pieces — unchanged from version 1 |
| palochka | always ӏ U+04CF (not Latin I, not Ӏ U+04C0) — normalize your input |
| decoding | beam 4 (greedy: −0.5 / −0.6 chrF, about 9× faster with the int8 ONNX on a CPU) |
| licence | CC-BY-NC-4.0 |
Usage
from transformers import MarianMTModel, MarianTokenizer
name = "kubataba/ru-kbd-bidirectional" # version 2; revision="v1" for version 1
tok = MarianTokenizer.from_pretrained(name)
model = MarianMTModel.from_pretrained(name).eval()
def translate(text, target): # target: "kbd" or "ru"
batch = tok([f">>{target}<< {text}"], return_tensors="pt")
out = model.generate(**batch, num_beams=4, max_new_tokens=160)
return tok.decode(out[0], skip_special_tokens=True)
print(translate("Сейчас он работает в Москве.", "kbd")) # Иджыпсту ар Москва щолажьэ
print(translate("Иджы псори нэгъуэщӏущ.", "ru")) # Теперь все по другому.
Write the Kabardian palochka as ӏ (U+04CF). Inputs with Latin I or 1 in its place should be normalized first.
Recommended use: translate sentence by sentence (whole paragraphs lose content), and stop the generation if a short piece repeats four times in a row (degenerate loops are rare but possible, mostly in the int8 version).
Results
FLORES-200 (devtest, first 200 sentences, Russian side from FLORES) — the same evaluation set as for version 1. The 200 Kabardian references were made for this evaluation with Yandex Translate: sentences 1–50 were checked and corrected by a native speaker, sentences 51–200 are the machine translation used as is. A machine-made reference favours outputs that resemble that system, so treat the 51–200 part as indicative; the 1–50 column is the human-checked one. The references are not distributed. FLORES sentences were removed from all training data; the same 200 sentences were also used to choose checkpoints.
| model | ru→kbd (200) | kbd→ru (200) | ru→kbd (1–50, human-checked) | kbd→ru (1–50) |
|---|---|---|---|---|
| version 1, beam 4 | 55.9 | 49.6 | 56.4 | 50.1 |
| version 2, beam 4 | 59.6 | 51.3 | 60.2 | 52.8 |
| version 2, int8 ONNX, beam 4 | 59.6 | 51.3 | — | — |
| version 2, int8 ONNX, greedy | 59.1 | 50.7 | — | — |
anzorq test set (1 081 pairs, mostly dictionary entries — very short): ru→kbd 46.6, kbd→ru 42.5 (version 1: 45.8 / 40.5).
Hyphenated words keep their hyphen in 99.8% (kbd→ru) and 100% (ru→kbd) of a 500-sentence probe; Latin words in the source are carried through to the output in 98% (ru→kbd) and 96% (kbd→ru) of cases (46 Latin words in FLORES).
Training
Base model and tokenizer. The transformer body comes from Helsinki-NLP/opus-mt-zle-zle, a multilingual East
Slavic model; it replaced the Russian–Ukrainian body of version 1. The joint Russian–Kabardian tokenizer of
version 1 was kept, so the model files have the same format. Each piece's embedding started as the mean of the base
model's embeddings of the pieces the base tokenizer split it into; the direction tags started from the base model's
own tags (>>ru<< from >>rus<<, >>kbd<< from >>bel<<).
Parallel data. 322 582 Russian–Kabardian pairs: our cleaned compilation from
adiga-ai/circassian-parallel-corpus and
adiga-ai/circassian-russian_texts_synthetic.
Compared with version 1, lines whose Russian is the machine translation of the synthetic texts were removed, lines
flagged by our defect detectors (copies, calques, unsupported press names, repeats) were removed, sentences of the
parallel corpus dropped by an earlier length filter were returned through the same checks, and hyphenated words split
by the source were restored. Each pair is used in both directions, twice per epoch.
Synthetic data — the target side is always real text. Machine-made text is only ever on the input side, so the model learns to produce real Kabardian and real Russian.
- Real Kabardian with machine Russian (teaches ru→kbd): from version 2 of
kubataba/kbd-ru-synthetic-backtranslation; 547 504 pairs in the final stage. - Real Russian with machine Kabardian (teaches kbd→ru): from version 2 of
kubataba/ru-kbd-synthetic-forward-translation; 584 358 pairs in the final stage.
For the final stages both synthetic sides were re-translated by earlier versions of this translator. The Kabardian side of the forward-translated set was re-translated by three earlier versions, and for each sentence one candidate was kept, primarily the one with the fewest Russian words left over from the source: Russian lemmas left in that side fell from 44.5 to 18.7 per 100 sentences. The Russian side of the back-translated set was re-translated by an earlier version as well. Programme listings and the numbering of crossword clues were removed, and pairs with lost numbers or truncated output were dropped. The re-translated versions are not published.
84 000 steps in total (batch 128, RTX 4090, about 2.5 hours):
| steps | what was trained | data |
|---|---|---|
| 1 000 | the new vocabulary embeddings only | corpus v2 |
| 31 000 | the whole model | corpus v2: parallel pairs twice, synthetic pairs, Latin carry-through |
| 30 000 | the whole model | the cleaned corpus v2 |
| 1 000 | the whole model | parallel pairs and Latin carry-through only |
| 21 000 | the whole model | parallel pairs twice, re-translated synthetic sides |
Hyperparameters: AdamW, weight decay 0.01, label smoothing 0.1, inverse square-root schedule, batch 128, max length 128 tokens; lr 3e-4 for the first 32 000 steps, then 1e-4, and 3e-5 for the parallel-only stage; bf16 on the GPU. The final checkpoint was chosen by beam-4 chrF on FLORES-200 and the anzorq test set and by listening to a translated book chapter.
Limitations
- Quoted text. In the Kabardian press, names in quotes stay in Russian, and the model learned to copy quoted text. For direct speech in quotes, translate the inside of the quotes separately and put the quotes back (in our app: quoted spans of 4+ words are translated apart; shorter ones lose the quotes and are translated in context).
- Rare words (technical, archaic) may be left in Russian; a dictionary substitution is not a fix — Kabardian needs the inflected form.
- kbd→ru improved less than ru→kbd; short isolated words remain the hardest case in both directions.
- Long sentences (> 40 words) are rare in training; split at clause boundaries.
Licence and attribution
The model is released under CC-BY-NC-4.0 (non-commercial). The copyright holder uses it in the SayFable app.
- Base model:
Helsinki-NLP/opus-mt-zle-zle(Apache-2.0). Tokenizer: version 1 of this model. - Parallel data: Anzor Qunash (adiga.ai), Circassian–Russian Parallel Corpus (CC-BY-4.0) and Circassian–Russian texts (MIT).
- Monolingual Kabardian:
anzorq/kbd_monolingual(MIT), texts of adyghepsale.ru, apkbr.ru, adygabza.ru, cherkes-haky.ru. - Monolingual Russian: Russian Wikipedia and Wikisource (CC-BY-SA); 19th-century Russian fiction from
nevmenandr/accentual-syllabic-verse-in-russian-prose(MIT; B. V. Orekhov, 462 texts). - Evaluation: FLORES-200 (CC-BY-SA-4.0).