Kyrgyz Morphological Tagger — released checkpoint
The seed-42 checkpoint of a morphological tagger for Kyrgyz: it predicts UPOS, a Kyrgyz-specific XPOS, the lemma, and the full Universal Dependencies feature bundle for every token. Syntax (HEAD/DEPREL) is not predicted.
This repository holds weights only. The code that runs them is on PyPI:
pip install kyrgyz-morph-tagger
from kyrgyz_morph_tagger import Tagger
tagger = Tagger.load() # fetches this checkpoint
for sentence in tagger.tag("Мектептерибизден эрте келдик."):
for token in sentence:
print(token.form, token.lemma, token.upos, token.feats_string)
Мектептерибизден Мектептер NOUN Case=Abl|Number=Plur|Poss=Yes
эрте эрте ADV _
келдик кел VERB Mood=Ind|Number=Plur|Person=1|Polarity=Pos|Tense=Past|Voice=Act
. . PUNCT _
Architecture
A faithful PyTorch re-implementation of UDPipe 2: per word, a character-level BiGRU, a
trainable word embedding, and frozen contextual embeddings from xlm-roberta-base (mean of
the last four layers, mean subword pooling) feed a 2-layer BiLSTM, with four classification
heads — UPOS, XPOS, a lemma edit-rule, and a monolithic FEATS label.
The encoder is frozen, so at inference the package also downloads xlm-roberta-base through
transformers. No architectural novelty is claimed: this is a strong, reproducible baseline for
the companion corpus.
Accuracy
Test set of the Kyrgyz UD corpus (425 sentences), exact match, gold tokenization:
| UPOS | XPOS | UFeats | AllTags | Lemmas | |
|---|---|---|---|---|---|
| This checkpoint (seed 42) | 93.90 | 92.63 | 81.25 | 79.44 | 87.96 |
| Mean ± std over 4 seeds | 93.80 ±0.12 | 92.53 ±0.13 | 80.96 ±0.33 | 79.19 ±0.40 | 88.06 ±0.14 |
UFeats is exact-set match of the whole feature bundle; a single wrong feature invalidates the
token. That is the honest measure of full morphological analysis in an agglutinative language,
and it is why UFeats (81) sits well below UPOS (94).
Under the same exact-match metric, three frontier LLMs given the same tagging scheme in the prompt reach UFeats 58–60 zero-shot, against this tagger's 79.6 on the same subsample.
Training data
The Kyrgyz UD treebank — 8,485 sentences (~98k tokens) with manually annotated gold morphology, split 90/5/5. The corpus is CC BY-SA 4.0 and builds on UD_Kyrgyz-KTMU and UD_Kyrgyz-TueCL.
Licence
These weights are released under the PolyForm Noncommercial 1.0.0 licence: free for research, teaching, personal use and noncommercial organisations. Commercial use requires a separate licence from the licensor. This is not an OSI-approved open-source licence.
The training corpus is licensed separately (CC BY-SA 4.0) and is not covered by this licence.
Limitations
- Morphology only — no syntax.
- Trained on news, web and literary text; other domains will be weaker.
- Raw-text input goes through a rule-based tokenizer (token F1 99.3, 95.3 % of sentences exactly
right on the test set). If you already have tokens, use
tag_pretokenized()to keep the tokenizer out of the loop. - Sentences longer than 512 XLM-R subwords lose contextual embeddings for the overflow; the package warns when this happens.
Links
- Code and experiments: https://github.com/AlibekovNurtilek/kyrgyz-morph-tagger
- Corpus: https://github.com/AlibekovNurtilek/kyrgyz-ud-treebank