diactag-1.0

A diacritic model that cannot corrupt your text.

DiacTag restores accents and tone marks in 10 languages by classifying each character rather than generating new text — so the output is guaranteed to be the input with marks added, and nothing else.

Input:   se eranko naa si gbo o?
Output:  ṣé ẹranko náà sì gbọ́ ọ?

37.6M parameters. Runs on CPU. Compliance 1.0000 by construction.

📖 Full documentation

Install

pip install olaverse[deeplearning]    # PyTorch checkpoint
pip install olaverse[onnx]            # adds the int8 ONNX backend

from olaverse.nlp import Diacritizer

d = Diacritizer(model="diactag-1.0", lang="yo")
d.restore("se eranko naa si gbo o?")
# → 'ṣé ẹranko náà sì gbọ́ ọ?'

Loading goes through olaverse.nlp. The architectures field in config.json documents what the checkpoint is, but there is no auto_map and no remote modeling code, so AutoModel.from_pretrained will not resolve DiacTagger — that is deliberate, not an omission.

lang= takes ISO-639-1 ("yo") or ISO-639-3 ("yor"). Leave it out and the model's own LID head decides:

d = Diacritizer(model="diactag-1.0")
d.restore("Co ay rat dam dang")        # → 'Cô ấy rất đảm đang'
d.detect_language("Lodz jest piekna")  # → ('pol', 0.9999)

How it works

The output has exactly as many characters as the input, in the same order, with the same base letters. Only the marks change. So don't generate — classify.

 input      s      e      r      a      n      k      o
            │      │      │      │      │      │      │
            ▼      ▼      ▼      ▼      ▼      ▼      ▼
     ╔═══════════════════════════════════════════════════╗
     ║          character transformer encoder            ║
     ╚═══════════════════════════════════════════════════╝
            │      │      │      │      │      │      │
   SHAPE   DOT     ·      ·      ·      ·      ·      ·
   TONE     ·      ·      ·    ACUTE    ·      ·      ·
            │      │      │      │      │      │      │
            ▼      ▼      ▼      ▼      ▼      ▼      ▼
 output     ṣ      e      r      á      n      k      o
            └──────┴──────┴──────┴──────┴──────┴──────┘
              base letters COPIED, never predicted

The base character is not an output of the network, so strip(output) == strip(input) holds for a trained model, an untrained one, or the int8 export. The SDK asserts it on every call rather than assuming it.

Architecture

      graphemes  [B,T]              language id  [B]
           │                              │
           ▼                              ▼
    ┌─────────────┐                ┌─────────────┐
    │  char emb   │                │  lang emb   │   11th slot =
    │   403×512   │                │    11×512   │   "unknown language"
    └──────┬──────┘                └──────┬──────┘
           │                              │
           └──────────────► + ◄───────────┘   additive: no <yor> token,
                            │                 no position spent
                            ▼
        ╔═══════════════════════════════════════╗
        ║          × 12  encoder block          ║
        ║  ┌─────────────────────────────────┐  ║   bidirectional —
        ║  │ RMSNorm → MHA(8 heads, RoPE)    │  ║   a diacritic depends on
        ║  │ + residual                      │  ║   what follows as much
        ║  │ RMSNorm → SwiGLU 512→1344→512   │  ║   as what precedes
        ║  │ + residual                      │  ║
        ║  └─────────────────────────────────┘  ║
        ╚═══════════════════╤═══════════════════╝
                            │  H  [B, T, 512]
        ┌──────────┬────────┴────────┬───────────┐
        ▼          ▼                 ▼           ▼
   ┌─────────┐┌─────────┐      ┌──────────┐┌──────────┐
   │  SHAPE  ││  TONE   │      │   MLM    ││   LID    │
   │ 512→15  ││ 512→8   │      │ 512→403  ││ 512→10   │
   │per char ││per char │      │ tied emb ││mean-pool │
   └────┬────┘└────┬────┘      └────┬─────┘└────┬─────┘
        │          │                │           │
        └────┬─────┘           training only  language
             ▼
      legality mask  →  argmax  →  compose(base, shape, tone)

The task heads together are 12k parameters — 0.03% of the model. Nearly everything is shared encoder, which is why ten languages cost roughly the same as one.

Shape and tone are separate, and language-dependent

Every character factorizes as base + SHAPE + TONE:

   ẹ́   →   e  +  DOT_BELOW   +  acute        (Yorùbá)
   ộ   →   o  +  CIRCUMFLEX  +  dot-below    (Vietnamese)
   ữ   →   u  +  HORN        +  tilde        (Vietnamese)
   ń   →   n  +     —        +  acute        (Polish)

SHAPE changes letter identity. TONE changes pitch or stress. Which marks belong to which slot is declared per language, and that is the point:

   U+0323  dot below      →  SHAPE in Yorùbá     (ẹ and e are different letters)
                          →  TONE  in Vietnamese (nặng, one of six tones)

   U+0303  tilde          →  SHAPE in Spanish    (ñ is a letter)
                          →  TONE  in Vietnamese (ngã)

A flat label set can't express that — it sees ọ and assigns one class, blind to the fact that the codepoint plays different grammatical roles in the two languages. Factorized, Vietnamese collapses from ~60 flat classes to 5 shapes × 6 tones, every component seen thousands of times. Across all ten languages the space is 15 shapes and 8 tones.

Results

diacbench, 1000 sentences per language.

Against the model it replaces: Yorùbá 0.2006 → 0.0836 (58% lower), Hausa 0.0593 → 0.0041 (93%). Much of the Hausa gain is the compliance guarantee rather than better modelling — a large share of the old error was text corruption, not wrong accents.

Versus diacnet-1.1

Versus frontier LLMs

300 sentences per language, matching how the baselines were run. These are not the same numbers as the table above, which uses all 1000; Yorùbá alone moves ~1pp between the two samples.

Best on 7 of 10; Vietnamese and Portuguese genuinely lose.

Those LLM numbers are the charitable ones — from a harness that discards any output no longer stripping back to the input. Raw, against the floor of simply copying the input unchanged:

On three languages a frontier model makes the text worse than doing nothing. Hausa is already 97.6% correct if left alone; Claude returns it 35% wrong. That fallback harness is this architecture, reimplemented externally. Here there is nothing to discard.

Training

Data. 7.0M sentences (web + Wikipedia) plus 131k from a Q&A passage set — the only non-encyclopedic register in the mix — gated on diacritic density and deduplicated against the eval splits by stripped-form hash.

The corpus was under-marked, and it was teaching the model to under-mark. Yorùbá density is bimodal: a mode at 0.10 holding 89% of sentences, and one at 0.43 holding 11%. Correctly marked Yorùbá lands around 0.30–0.45 — so the majority of the data was under-marked text being used as ground truth. No other language shows this (peak prominence 0.55 for Yorùbá, below 0.10 for every other).

The fix isn't deletion. Under-marked Yorùbá is still Yorùbá — good data for the language, poisonous for the marks. Sentences below the detected trough (0.2925) keep their masked-character targets and contribute no diacritic supervision. The MLM head trains on all 1.9M Yorùbá sentences; the tagger heads on the clean ~205k.

Also filtered: non-Latin script (Ajami in Hausa Wikipedia, CJK and Devanagari names elsewhere) and cross-language contamination. Without both, the label space inflates from 15 shape classes to 99, most of them junk competing in the same softmax.

Objective.

loss = 1.0·CE(shape) + 1.0·CE(tone) + 0.3·CE(masked_char) + 0.05·CE(language)

The masked-character head is auxiliary, dropped at inference. Its target is always the fully marked character — masking ú and accepting u would teach the encoder that bare vowels are valid Yorùbá, which is the failure being fixed.

Hyperparameters. AdamW, lr 3e-4, 2000 warmup, cosine decay to 5%, weight_decay 0.05, batch 128 × 2 accumulation = 256, seq len 256, bf16, 120k steps. ~13 hours on one A100. Language sampling temperature-balanced at α=0.5. Augmentation: partial_mark_prob=0.30, case perturbation, 3% protected-span injection, mlm_prob=0.15, lang_dropout=0.12.

Deployment

d = Diacritizer(model="diactag-1.0", lang="yor", onnx=True)

Three times faster and four times smaller for +0.03pp DER. Compliance stays 1.0000 under quantisation — the guarantee is architectural, not a property of numeric precision. A 38MB artifact at 244 chars/s on one CPU core takes the GPU off the serving bill entirely.

End-to-end figures: one sentence at a time, full pipeline, single core. Use restore_batch for throughput workloads. p95 is 788–1099 ms across all backends — the tail is length-driven, not backend-driven, so quantisation doesn't fix it.

Abstention

At 0.90: 97% of characters restored at 99.6% accuracy, the rest flagged. A wrong tone mark changes meaning; a missing one is merely incomplete. Threshold is per-request, so one loaded model serves a CMS pre-fill and a legal pipeline at different points on the same curve.

text, details = d.restore(src, return_details=True)
review = [c for c in details if c.confidence < 0.9]

Confidence is temperature-calibrated on validation (T = 1.14).

Lexicon reranking (opt-in, off by default)

If the model emits a non-word whose stripped form has attested variants, rescore the candidates under the model's own distribution. It never invents a word.

d = Diacritizer(model="diactag-1.0", lang="yor", use_lexicon=True)

Measure it on your data before enabling it. On diacbench it cuts non-word outputs by 27% and raises Yorùbá DER by 15% (0.0836 → 0.0961). The cause is our own density gating: restricting the lexicon to well-marked text shrank the Yorùbá vocabulary from 86k forms to 18k, so "not in the lexicon" often means "rare or inflected word we didn't keep," and correct outputs get overwritten. Off by default.

Limitations

• No typo correction. Cannot insert or delete characters, so it can't fix Ile → Ilé and teh → the in one pass. The price of the guarantee. The per-character confidence is a natural trigger for a separate corrector.
• Yorùbá is still hard. DER 0.0836, 83% of it tone direction. Sentence-level exact match is 0.084 — 92 of every 100 Yorùbá sentences contain at least one wrong mark, a harsher and truer framing than DER.
• Igbo and Hausa tone numbers are not achievements. tone_DER of 0.0013 and 0.0001 looks superb and means little: those orthographies barely write tone. There was almost nothing to learn.
• Dense input degrades. The Polish pangram Zażółć gęślą jaźń has density 0.565 against a Polish median of 0.068 — nine times denser than real Polish — and the model misses six characters, despite a Polish DER of 0.0022.
• Some errors are irreducible. Viaggio and Viaggiò are both valid Italian and the stripped form contains no information distinguishing them.
• Fixed label space. Adding a language with new marks invalidates existing checkpoints. SPEC_VERSION is checked on load, so a mismatch fails loudly.
• Not a diacnet replacement. A parallel architecture with a different contract. diacnet-1.1 remains the option where generation is wanted, and wins on Vietnamese and Portuguese.

Which model should I use?

Files

Citation

@misc{diactag10,
  title  = {DiacTag: diacritic restoration as constrained sequence labeling},
  author = {Olaverse Labs},
  year   = {2026},
  url    = {https://huggingface.co/olaverse/diactag-1.0}
}

Built by Olaverse Labs · Docs · Benchmarks · Apache 2.0