HyeBERT (fine-tuned) — Armenian Participle-Clause Punctuation
Fine-tuned Armenian-specific aking11/hyebert (6-layer, ~66M params) for
punctuation of Eastern Armenian participle clauses, as 4-class token labeling.
From the CODASSCA 2026 paper Sequence Labeling for Low-Resource Syntax.
Labels
0 O · 1 COMMA_AFTER · 2 BUTH_AFTER · 3 REMOVE_COMMA
Results (macro-F1)
| Benchmark | macro-F1 |
|---|---|
| Shtemaran 292 (clean textbook) | 0.3260 |
Full fine-tuning (no frozen layers), sqrt inverse-frequency class weighting.
In the paper, the deeper 12-layer mBERT outperformed this model, illustrating
that model depth outweighed language-specificity for this task. Raw checkpoint
and configs are in artifacts/; the notebook is in training/.
Usage
from transformers import AutoTokenizer, AutoModelForTokenClassification
tok = AutoTokenizer.from_pretrained("AlbertHakobyan/hyebert-armenian-participle-punct")
model = AutoModelForTokenClassification.from_pretrained("AlbertHakobyan/hyebert-armenian-participle-punct")