Lithuanian morphology tagger — accentuation flavor
Token-classification tagger built as the disambiguation engine for Lithuanian accentuation (kirčiavimas) pipelines, notably Kirčiuoklė. It emits UPOS + agreement features in the label space of Lithuanian traditional grammar — the space used by accentuation dictionaries (VDU convention): DET is folded into PRON and AUX into VERB (Lithuanian grammar has neither category; determiners are įvardžiai, auxiliaries are ordinary veiksmažodžiai). If you want strict UD categories, use the -morphology or -morphology-full siblings instead.
Benchmarks (measured)
End-to-end accentuation quality — agreement of homograph stress choices with the production VDU+UDPipe pipeline (each ambiguous word, e.g. galvõs vs gálvos, resolved by this model's morphology):
| eval | choices agreeing |
|---|---|
| 766-char sample text (20 ambiguous words) | 20/20 |
| 2,600-token Wikipedia corpus (370 ambiguous words) | 323/370 (86.5%) — equal to an internal UDPipe-taught reference; hand-checked disagreements split both ways |
Tag-level (full 684-sentence ALKSNIS gold test, traditional-grammar projection): slots 86.4%, UPOS 90.6%, at 880 tok/s (ONNX INT8, CPU). For treebank-style tagging the -morphology sibling scores higher — this flavor exists for accentuation, where its conventions match the dictionary being scored against.
Not emitted by design: DET/AUX categories, FEATS beyond the ten agreement keys, lemmas (LEMMA = lowercased form; consumers apply a yra→būti rule for the copula homograph).
Usage (verified)
import torch
from transformers import AutoTokenizer, AutoModelForTokenClassification
repo = "alexbalandi/litlat-bert-lithuanian-morphology-vdu"
tok = AutoTokenizer.from_pretrained(repo)
model = AutoModelForTokenClassification.from_pretrained(repo)
words = ["Tas", "namas", "yra", "gražus"]
enc = tok(words, is_split_into_words=True, return_tensors="pt")
with torch.inference_mode():
logits = model(**enc).logits[0]
seen = set()
for pos, wid in enumerate(enc.word_ids(0)):
if wid is not None and wid not in seen:
seen.add(wid)
print(words[wid], model.config.id2label[int(logits[pos].argmax())])
# Tas PRON|Case=Nom|Gender=Masc|Number=Sing|Person=3 <- DET folded into PRON
# ...
An onnx/ folder with an INT8 model is included; a UDPipe-REST-protocol
serving container lives in the
project repo (local/tagger-hf/).
Training data & lineage
Gold corpora only, no NC-licensed model anywhere in the lineage: MATAS v3.0 (CC BY 4.0; UD features reconstructed from its Jablonskis XPOS) + UD_Lithuanian-ALKSNIS (CC BY-SA 4.0), gaps filled by iterated constrained-decoding self-training. Labels normalized to traditional-grammar categories in all splits.
License & attribution
CC BY-SA 4.0 (inherited from litlat-bert and ALKSNIS). Please credit: MATAS v3.0 (Rimkutė, Bielinskienė, Boizou, Dadurkevičius, Kovalevskaitė, Utka; CLARIN-LT), UD_Lithuanian-ALKSNIS (VDU), litlat-bert (Ulčar & Robnik-Šikonja, EMBEDDIA).