kmynas-parakeet-lt-v4 — Lithuanian ASR that writes punctuation

A parakeet-tdt-0.6b-v3 fine-tune on 3,087 hours of LIEPA-3, 0.6 B parameters, a TDT transducer. Two things separate it from the other open Lithuanian models:

• It writes punctuation, capitalisation and loanword orthography directly — no post-processing stage. „Kitas gyvenimas“, Boeing'u, Messenger'ių, sentence case, question marks. Models trained on raw LIEPA-3 output none of that, because LIEPA-3 has none of it; the convention here was built into the targets instead (see below).
• It is a transducer, not an encoder-decoder. Alignment is monotonic over frames, so it cannot skip a window or loop unboundedly. In our own reading of 67 minutes of real recordings, canary-1b-v2 emitted nothing for 197 s of clearly audible speech and repeated one word 90 times; whisper-based models hallucinate fluent sentences over silence. This architecture fails differently: messier text, but the words stay.

Speed, measured on a MacBook with no server involved: 30 s of audio decodes in 0.7 s on MPS (42× faster than realtime) and 1.1 s on CPU alone (27×). On the same machine azuolas-whisper-lt runs at roughly 4–5× realtime and canary-1b-v2 (int8 ONNX) at ~9×. An hour of recording takes about 90 seconds here.

For datacentre numbers, the Ąžuolas authors benchmarked our previous version on an RTX PRO 6000 and reported 819× realtime for kmynas v3 against 26× for Ąžuolas and 83× for Paprika — their measurement, not ours. Our own earlier run of a parakeet-LT model on an RTX 4090 gave 87–149× at batch 1, so treat the spread as a batching difference rather than a contradiction.

Training data

LIEPA-3 only (CC BY 4.0, Vilnius University / raštija.lt — thank you). No web audio, no synthetic speech, no other corpus.

Held-out speakers are excluded, and for read speech the held-out speakers' sentences are excluded too — read prompts are shared across speakers, so speaker-level holdout alone leaks the text. An earlier finding worth repeating: LIEPA-3 ships no test split, and a naive "first N speakers" holdout overlaps the read set.

The targets are LLM-normalised, and that is the point

Every target text went through one LLM call per 200 rows at medium reasoning effort, doing punctuation, capitalisation and loanword normalisation in a single pass (~2.5 M rows). Guards, because an LLM editing 2.5 M transcripts silently is a data-loss risk:

• Declared-change gate. The model must list every word it changed; applying the declared changes to the raw words must reproduce its own output token for token. Rows that fail are rejected, not accepted-and-hoped.
• Deterministic post-pass afterwards (no LLM): reverts digits (numbers stay words), hybrid derivations, Russian-origin names that Lithuanian transliterates, and applies the dominant mapping for lowercase loan terms consistently across the corpus.
• Dialect reverts. The dialect-notation conversion had also replaced spoken words with standard synonyms (nu→na, ale→bet, baba→močiutė, ana→ji) in 7.6 % of rows. The audio says the dialect word, so 7,900 words were put back position-wise against the original notation.
• Content-keyed cache, atomic saves, restartable under launchd — the pass took ~40 h.

Convention: loanwords keep their original spelling with an apostrophe before the Lithuanian ending (Facebook'e, Boeing'u), spelled-out acronyms become uppercase, person names stay adapted (Trampas), numbers stay words.

Augmentation

Earlier versions lost 2–5× WER when a clip had 0.5 s of quiet at an edge — training clips had 20–60 ms margins, real recordings do not. So: 0–800 ms of low-level noise or silence at clip edges on 40 % of rows plus ±6 dB gain at harvest time, and NeMo white-noise/gain augmentors during training.

Recipe

Trained from the pinned base (541d1f99) with a 4-piece vocab splice adding „ “ – ▁„, spliced at index V so the existing decoder/joint weights survive rather than being reinitialised. Effective batch 256 (8 × 32 accumulation), LR 5.5e-4 chosen by a measured probe rather than extrapolation, cosine decay, 1,500 warmup, MAX_DUR 20 s, bf16, 30,000 steps ≈ 3.04 epochs on one A100 80 GB, ~50 h.

Numbers, and why they understate the model

Scored on held-out LIEPA-3 (stripped WER, best of 12 checkpoints — the final one):

Read those as agreement with a convention, not as quality. The references are LIEPA-3's own transcripts, which record what was heard — including clipped colloquial endings (daryt, jum, pasirinkot). This model, trained on 898 h of read speech among everything else, writes the standard forms (daryti, jums, pasirinkote). Every one of those counts as a substitution. The effect grows with training: it is barely present at step 5,000 and dominant by step 20,000, so a lower WER can mean a less-trained model. On our own recordings the ranking inverts: agreement with an independent commercial transcript rose 85.0 % → 87.9 % → 91.1 % from step 5k → 15k → 20k while the LIEPA-scored WER moved far less.

Three more reasons a single WER number will mislead you here:

1. Punctuation is scored or not scored, and it changes everything. The same checkpoint is 13.44 stripped and 22.02 with punctuation compared verbatim. A model that emits no punctuation at all cannot lose those points.
2. Digits. Benchmarks like FLEURS print 25 where the speaker said dvidešimt penki. That is worth roughly 4 WER, and it says nothing about recognition.
3. Reference quality. LIEPA-3 references are truncated and colloquially clipped in places, so a model is marked wrong for writing correct Lithuanian.

We publish the numbers because withholding them is worse, but the honest evaluation was reading three of our own recordings — a book presentation, a sales call and a two-person work call — against the audio.

Known limitations

• Proper names drift between mentions of the same name. In one 7-minute recording it writes Šarpnickis consistently (3 of 3) but the same surname appears as both Leskauskas and Liskauskas in different sentences. A decode-time boosting list fixes this class of error far more cheaply than more audio does.
• Foreign technical vocabulary is mostly handled, but not reliably. At 30,000 steps a live sales call comes out as subscription, subscription'us, subscription'ų and Messenger'į — the convention applied correctly to an English noun being inflected in Lithuanian. Earlier checkpoints of the same file wrote "subskripti" and "subscripšinus", so this improved late in training and may still fail on rarer terms; Claude still comes out as "klodas".
• Verbatim vs readable is a choice we made one way, and it is not uniform: the model standardises clipped endings in some passages (pasirinkot → pasirinkote) and keeps them in others. If you need strictly verbatim output for legal or research transcription, test it on your own audio first.
• No diarization and no speaker labels.
• Trained on ≤20 s clips; it handles long audio natively (full attention), but very long files are better cut at pauses.

Usage

The repo carries the model in both formats: transformers weights at the root and the original kmynas-v4-last.nemo.

from transformers import pipeline          # transformers >= 5.14, which has ParakeetForTDT
asr = pipeline("automatic-speech-recognition",
               model="kristijonas/kmynas-parakeet-lt-v4", device="mps")   # or "cuda" / "cpu"
print(asr("pokalbis.wav")["text"])

import nemo.collections.asr as nemo_asr    # the original checkpoint, for a NeMo pipeline
m = nemo_asr.models.ASRModel.restore_from("kmynas-v4-last.nemo")
print(m.transcribe(["pokalbis.wav"])[0].text)

16 kHz mono. Output already carries punctuation and casing — do not run a punctuation restoration stage on top of it.

Timestamps (NeMo path)

The TDT decoder predicts durations, so word, segment and character timestamps come out of the same pass at no extra cost:

out = m.transcribe(["pokalbis.wav"], timestamps=True)
for w in out[0].timestamp["word"][:3]:
    print(w["word"], w["start"], w["end"])
for seg in out[0].timestamp["segment"][:3]:
    print(seg["start"], seg["segment"])

On a 7-minute recording that is 758 word entries and 32 segments, segmented at sentence ends — usable directly for subtitles or for lining a transcript up against audio.

Attribution

Base model nvidia/parakeet-tdt-0.6b-v3 (NVIDIA, CC BY 4.0). Data: LIEPA-3 garsynas (Vilnius University, raštija.lt, CC BY 4.0).