marian-uk-verbalizer
A 52.7M-parameter Marian sequence-to-sequence model that converts written Ukrainian text into the form it should be spoken — the text-normalization front end for a Ukrainian TTS pipeline. It expands Roman numerals (with correct case/gender agreement), dates, currencies, coordinates, phone numbers, IBANs, card numbers, math expressions, symbols, abbreviations and acronyms into spoken words, while leaving ordinary prose untouched.
Потрібно прочитати XX розділ до понеділка.
-> Потрібно прочитати двадцятий розділ до понеділка.
Я пропрацював у ФБР двадцять років.
-> Я пропрацював у еф-бе-ер двадцять років.
Full training/evaluation code, the router this model is meant to sit behind, and per-domain evaluation scripts: GitHub repository.
A quantized int8 CTranslate2 export of this same checkpoint (52 MB, ~99.1%
output-identical) is published separately at
aloudreader/marian-uk-verbalizer-ct2-int8.
Model description
- Architecture: MarianMTModel, 6 encoder + 6 decoder layers,
d_model=512, 8 attention heads, 2048 FFN dim — 52.7M parameters. - Tokenizer: shared 16k-token SentencePiece vocabulary (source and target).
- Decoder start token is
<s>(token id 0), not the usual pad token — this matters if you re-export the model (see the GitHub repo's CTranslate2 conversion notes). - Trained with
input_preprocessing: model-routing-v1: the model always receives the original, unmodified sentence and its output is used raw — no rule-based pre- or post-processing anywhere in the pipeline. This is a hard project constraint, not an implementation detail: seeAGENT.mdin the GitHub repository.
Intended use
Sits behind a lightweight shape-based router that sends plain Cyrillic prose
straight through and everything else (digits, Latin letters, symbols,
all-caps runs, …) to this model. Feeding the model plain prose it doesn't
need to touch is safe — the training data includes plenty of identity
(unchanged) targets — but the router avoids the extra inference cost. The
router implementation (regex/shape only, no word lists) is in the GitHub
repository at src/verbalizer/text.py.
This model is Ukrainian-specific and was trained and evaluated on general prose plus template-generated numeric/structured text. It is not intended for other languages or for tasks other than pre-TTS text normalization.
How to use
from transformers import MarianMTModel, MarianTokenizer
tokenizer = MarianTokenizer.from_pretrained("aloudreader/marian-uk-verbalizer")
model = MarianMTModel.from_pretrained("aloudreader/marian-uk-verbalizer")
text = "Потрібно прочитати XX розділ до понеділка."
batch = tokenizer([text], return_tensors="pt", padding=True)
out = model.generate(**batch, max_new_tokens=128, num_beams=1)
print(tokenizer.batch_decode(out, skip_special_tokens=True)[0])
Greedy decoding (num_beams=1) is what this checkpoint was evaluated with;
beam search is supported but was not found to improve the benchmark scores
enough to justify the extra latency.
Training data
- The full
skypro1111/uk-text-normalizationdataset (a 3,597-row slice is frozen out as an evaluation-only benchmark and never trained on). - Synthetic rows generated offline with
num2wordsandpymorphy3for domains and grammatical cases the base dataset covers thinly — most recently, Roman-numeral-plus-noun case/gender agreement across many governing verbs, prepositions and nouns per case, so the model learns to read the noun's ending rather than memorizing one preposition per case. Every generated target is cross-checked by parsing it back to a numeric value and comparing it against the source before it enters training.
No text extracted from books is included in or was used to build this model's public evaluation data (see the GitHub repository's data policy).
Evaluation
Frozen 3,597-row benchmark (held out of training, no data leakage):
| Metric | Score |
|---|---|
| Exact string match | 85.2% |
| Word error rate | 3.0% |
| Character error rate | 1.2% |
| Numeric value fidelity (every digit sequence read back and compared to source) | 100.0% |
| Digit sequence fidelity | 100.0% |
| Identity accuracy (plain-prose rows left untouched) | 100.0% |
Generated per-domain evaluation (lenient frame+slot scoring — the fixed part of each template sentence must match exactly, and each slot must be read as one of its accepted spoken forms):
| Domain | Correct | Rows |
|---|---|---|
| Coordinates | 100.0% | 500 |
| Phone numbers | 100.0% | 500 |
| Roman numeral + noun case agreement | 99.0% | 301 |
| Context (held-out book sentences, not published) | 97.4% | 500 |
| English words embedded in Ukrainian | 88.4% | 500 |
| Card numbers / IBAN | 86.8% | 500 |
| Acronyms (letter-by-letter reading) | 85.5% | 207 |
Limitations
- Acronym and card/IBAN reading are the weakest domains (~85–87%); most failures are single-letter substitutions in acronym spelling or a single misread digit group in long card numbers.
- The model has only ever seen Roman numerals I–XXXIX (by project convention,
L/C/D/Mare treated as ordinary letters — vitamin C, size L, flight D12 — since they collide with common non-numeral usage in Ukrainian text). - Like any seq2seq model, it can occasionally hallucinate or drop a token on out-of-distribution input; there is no rule-based safety net downstream — by design, its raw output is what gets spoken.
License
MIT. See the GitHub repository
for training/evaluation code under the same license. The
skypro1111/uk-text-normalization training dataset has its own license —
check it before redistributing data derived from this model.