Dewpoint

Punctuation and truecasing for speech-recognition output, in 79 languages. Speech recognisers hand back a stream of lowercase words with no punctuation, and Dewpoint restores it: it adds the commas, full stops and question marks, and the capitals. Open weights, MIT licence, free for everyone.

so i said meet at three thirty tuesday what do you think the iphone was announced by steve jobs in san francisco
→ So I said meet at three thirty Tuesday. What do you think? The iPhone was announced by Steve Jobs in San Francisco.

ich weiß nicht ob wir das schaffen aber wir sollten es versuchen was meinst du
→ Ich weiß nicht, ob wir das schaffen, aber wir sollten es versuchen. Was meinst du?

πού είναι ο σταθμός ξέρεις να μου πεις
→ Πού είναι ο σταθμός; Ξέρεις να μου πεις;

These are unedited outputs of this checkpoint. Dewpoint is a tagger, not a generator. For each word it decides which mark follows (none, , . ? !) and how the word is cased (lower, Capitalised, UPPER). It never adds, drops, reorders or rewrites a word, so it cannot invent content.

Results

On the IWSLT TED benchmark, Dewpoint scores higher than every other multilingual punctuation model we tested, under all three test conditions. On the two text test sets it is behind no other model in any language: 0 of 9 on the official set and 0 of 57 on the extended one. Under ASR noise it trails in 7 of 33, 5 of them to raw Whisper.

Macro-F1 over COMMA, PERIOD and QUESTION, averaged over languages. Every system runs through the same harness and gets the same dev-set threshold calibration.

Head to head on each model's own languages. The averages above include languages the other models were never trained on, which pulls their means down. Scored only on the languages each model lists on its own model card, the margins are smaller:

Dewpoint is ahead in every one of these languages except Bengali on Whisper ASR output against xlm-r truecase.

Each cell is that model's mean macro-F1 vs Dewpoint's on the same languages, with the number of languages in brackets. This is the fairer comparison.

Quick start

pip install torch transformers safetensors huggingface_hub numpy   # GPU or CPU
# or, with no torch at all:
pip install onnxruntime tokenizers huggingface_hub numpy

import os, sys
from huggingface_hub import hf_hub_download
sys.path.insert(0, os.path.dirname(hf_hub_download("valkayuh/dewpoint", "dewpoint.py")))
from dewpoint import Punctuator

p = Punctuator.from_pretrained("valkayuh/dewpoint")   # torch if installed, else ONNX
p.restore("what time is it in tokyo right now", lang="en")
# 'What time is it in Tokyo right now?'

from_pretrained downloads only what the chosen backend needs. The torch backend fetches the safetensors (3.5 GB, or 1.2 GB with members=["mmbert-base"]), the ONNX backend fetches the ONNX graphs, and neither fetches the other.

Always pass the language. It selects the calibrated decision thresholds, turns casing off for scripts that have none, and picks the right marks to write: ؟ for Arabic, ; for the Greek question mark, 。 and ， for Chinese, । for Hindi.

Single-model mode. Pass members=["mmbert-base"] to download and run only the 307M-parameter mmBERT-base half. It uses its own calibration, runs in about half the GPU time, and still beats every other model we tested on every benchmark:

p = Punctuator.from_pretrained("valkayuh/dewpoint", members=["mmbert-base"])

Streaming. For live captions, each word is released only once it has right-hand context and a stable label, so marks don't flicker:

s = p.stream(lang="en", lag=6)
for chunk in asr_chunks:            # partial transcripts as they arrive
    print(s.push(chunk), end="", flush=True)
print(s.finish())

Labels and probabilities. p.predict(words, lang) returns per-word labels plus the averaged posteriors (punct_probs, case_probs), in case you want confidence scores or your own decision rule.

Command line.

echo "i called the office this morning but nobody answered did you get my message" | python dewpoint.py --lang en
# I called the office this morning, but nobody answered. Did you get my message?

Formats and integration

ONNX graphs. There is one per member, in onnx/mmbert-base/ and onnx/xlm-roberta-large/ (the latter keeps its weights in model.onnx.data beside the graph). Both are fp32, opset 17, with dynamic batch and sequence axes:

• inputs: input_ids, attention_mask, int64 [batch, seq], from that member's tokenizer.json with the words passed pre-tokenised
• outputs: punct_logits [batch, seq, 5] and case_logits [batch, seq, 3]

Around the graph, dewpoint.py does four things, and a port needs the same four:

1. split the input into overlapping windows of at most 508 subwords, keeping each window's centre (plan_windows)
2. read each word's label from its first subword
3. average the two members' softmax posteriors
4. add the calibrated per-class bias from ensemble_config.json for the language, then take the argmax

Verified. Run over the whole official test set on CPU, the ONNX backend scores 0.7558 macro-F1 against the published 0.7559; the largest difference in any scored language is 0.0011, the gap between fp32 ONNX and the bf16 GPU run the published numbers come from. Its tokenisation matches transformers exactly on 323 documents in 57 languages.

Speed

Measured on a 584-word TED passage, after warm-up, with the code in this repo:

Input of any length is handled with overlapping 512-token windows. Only the centre of each window commits its predictions, because words near a window edge lack context and score measurably worse.

Model

Dewpoint is an ensemble of two dual-head token classifiers. Both members read the same word list and produce one posterior per word, and those posteriors are averaged.

Each member is an encoder, then a residual shared trunk (Linear, GELU, LayerNorm), then two linear heads:

• punctuation, 5 classes: O COMMA PERIOD QUESTION EXCLAM
• case, 3 classes: LOWER (meeting), CAP for a capitalised first letter (Tuesday, Tokyo, a sentence's first word), and UPPER for a word written entirely in capitals (NASA, BBC, DNA). Mixed-case words such as iPhone are handled by a word list, described below.

There is one decision per word, not per subword: the label sits on each word's first subword. Punctuation and case are learned together because they are correlated: a sentence end means a capital comes next.

Combining the members. Both members label the same words, so their outputs line up without any tokenizer reconciliation. The average is taken in probability space, not log space. With a geometric mean, one confident but wrong member could veto a class; the arithmetic mean degrades gracefully instead.

Calibration. Class weighting stops the model predicting "no punctuation" everywhere, but it buys recall at the cost of precision, so a plain argmax is not the F1-optimal rule. Instead, a per-class additive bias on the log-posterior is fitted on IWSLT dev2010, which is held out of every test table. Whether to add per-language biases on top, and how far to trust them, is chosen by cross-validation inside dev2010: its talks are split in half five times, and each strategy is fitted on one half and scored on the other. An unshrunk bias per language, fitted on eight talks, overfits: it was never the winning strategy, and for Dewpoint it scored worst. Dewpoint uses per-language biases pulled three quarters of the way back to the pooled one. The ensemble is calibrated as a system in its own right, and every other model in the tables went through the identical procedure, each choosing its own strategy.

Post-processing, in restore() only (predict() returns the raw decision):

• a capital after every sentence-final mark
• a gazetteer of 4,896 mixed-case forms such as iPhone, McDonald and GHz
• a closing mark if the text would otherwise end on none

Evaluation

• Three test conditions:
  • the IWSLT 2017 official tst2010–2015 sets, with 9 scorable languages
  • the WIT3 releases of the same 90 talks in 57 language tracks, so only the language changes
  • an ASR condition: TED text spoken by TTS, transcribed by Whisper large-v3-turbo, and aligned back to the reference, in 33 languages
• Metric. Macro-F1 over COMMA, PERIOD and QUESTION, averaged over languages. EXCLAM is folded into PERIOD, because no other model emits it. Accuracy is never reported: about 87% of words carry no mark, so a model that predicts nothing is "87% accurate".
• Exclusions. Japanese and Thai are excluded, and named here. Their TED references carry almost no punctuation (0.38% and 0.12% of tokens).
• Reference audit. Two errors in the reference's label maps were fixed for every system alike. Armenian writes its full stop as an ASCII colon; fixing that doubled every system's Armenian score. And the Burmese comma U+104A had been mapped to a full stop.
• No TED data was used for training. TED is the test set.

Every language on the extended set

Under ASR noise

5 of the 7 are to raw Whisper (English, Spanish, French, Italian, Portuguese). Whisper punctuates from the audio itself, so it can use pauses and intonation that a text-only tagger never sees. The rest: Bengali (−0.044 to fullstop-sonar), Albanian (−0.025 to xlm-r truecase). Languages marked * in the chart have under 3,000 scored words, so their gaps are noisy.

Per class, and truecasing

Mean per-class F1 against the strongest other model:

Truecasing F1 against the only other model that restores case, on cased scripts:

Reading the truecasing table. CAP scores words whose first letter should be a capital: names, places, and the first word of every sentence. UPPER scores words written entirely in capitals, which in practice means acronyms and initialisms such as NASA, BBC or DNA. These are much rarer, so they get their own score and are not swamped by ordinary capitals. A single capital letter such as I counts as CAP.

The truecasing comparison covers cased scripts only. On caseless scripts (Arabic, Chinese, Korean, Hindi and others) the case head is masked by design, so Latin loanwords inside them are never capitalised. xlm-roberta_punctuation_fullstop_truecase does capitalise them, so it wins on those words by construction. The native EXCLAM class scores 0.11 F1 on the official set and 0.07 on WIT3. No other model emits it at all.

Size

Where the gain comes from

Every other model in these tables is an XLM-RoBERTa model, so a win could come from the architecture or from the training recipe. To separate the two, we trained XLM-R-large, the encoder fullstop-punctuation-multilang-large uses, on this corpus with this recipe.

The recipe is worth +0.08 to +0.13 macro-F1. Swapping the encoder is worth −0.004 to +0.016. The corpus, the orthography fixes and the decision rule matter roughly an order of magnitude more than the encoder. Neither encoder dominated the other, and averaging exploits exactly that: members that do not make the same errors.

Per-language tables

All per-language, per-class scores for every system are in results/scores.json.

Training

Text: 79 languages, no TED

The corpus is temperature-sampled so that the largest languages don't swamp the rest. Labels come free: strip the punctuation and case from clean prose, and what was stripped is exactly what the model must predict. Synthetic ASR noise is applied to about two thirds of training examples: filler words, dropped short words, character substitutions, merged and split words, numbers written the way they're spoken, and truncated utterances.

Speech: round-trip ASR data

A slice of the training text was spoken aloud by TTS, transcribed by Whisper large-v3-turbo, and aligned back to the original. That gives inputs with genuine recognition errors but ground-truth labels. The corpus has 0.96M unique words across 23 languages, voiced by three permissively licensed engines. Oversampled to match the per-language dose of the original recipe, it makes up 6.8% of the final fine-tuning stage.

Every Piper voice's licence was read from its model card by code, and any voice whose card said non-commercial, share-alike, "see URL" or nothing at all was refused. The voice that produced a language's ASR test audio is never used for its training audio. French is the exception, because Kokoro has only one French voice. No audio was kept or is distributed.

Procedure

Each member was trained in stages. Every stage starts from the weights the previous stage ended with, and each later stage makes smaller, more targeted changes:

1. Learn the task. The encoder starts from its public pretrained weights, knowing language but nothing about punctuation, and learns to predict marks and capitals on a large clean-text corpus.
2. Rebalance toward speech. Training continues on a resample of the corpus that gives more weight to spoken-style sources (parliamentary transcripts, Tatoeba, Aya) and mixes short passages in with long ones, closer to what ASR produces.
3. Harden against recognition errors. A short final stage on a 55M-word mix that includes the round-trip ASR data above, at a low learning rate, so the model learns to punctuate text containing Whisper's mistakes without forgetting the rest.

How to read the table:

• Steps. A step is one update of the weights, computed from a batch of 64 text windows of up to 512 subword tokens each, roughly 30,000 tokens. The first mmBERT stage, for example, saw about a billion tokens.
• Peak learning rate. This is the largest step size in that stage's schedule. It falls from stage to stage because later stages refine rather than relearn.
• XLM-R-large has fewer stages because it went straight to the rebalanced corpus. It was first trained as a controlled comparison against mmBERT-base, and later became the second member.

Settings shared by all stages:

• AdamW, with a learning rate that warms up and then decays along a cosine curve
• a higher learning rate for the two heads than for the encoder
• class-weighted cross-entropy on both heads, so the rare marks are not drowned out by the many words that take none
• label smoothing 0.02
• for XLM-R-large only, a frozen embedding matrix

Everything was trained on a single RTX 5080 with 16 GB.

What did not work

The distillation result is why Dewpoint is an ensemble. Its robustness comes from the two members disagreeing on each input. A single model trained to mimic their average has no disagreement left to exploit. Two of these experiments were pre-registered: their success criteria were written down before the run.

Limitations

• Two forward passes. The ensemble has 2.8× the parameters of its mmBERT half, for +0.013 to +0.020 macro-F1. If latency matters more than that margin, use members=["mmbert-base"].
• Text only. Raw Whisper still wins in English, Spanish, French, Italian, Portuguese under ASR conditions, most likely because it hears pauses and intonation. Adding a tagger after ASR only helps if your ASR doesn't already punctuate.
• English is not state of the art against English-only systems. On English tst2011, Dewpoint scores 0.800 macro-F1 (COMMA 0.715, PERIOD 0.823, QUESTION 0.860) without having seen any TED text. The strongest published English-only systems report about 0.85 overall F1 on IWSLT2011; they were trained in-domain on TED and scored on a differently preprocessed version of the set.
• Weak languages. Kurdish (0.11), Burmese (0.44), Tamil (0.51), Urdu (0.53) are still low in absolute terms, even where Dewpoint leads the other models. In Kurdish (0.11 against 0.28) the ensemble is below its own mmBERT half, so averaging in probability space does not help everywhere.
• Japanese and Thai are unmeasured. Their TED references are nearly unpunctuated, so no trustworthy number exists. The model is trained on both.
• Very short inputs. With only a few words there is little context, and the case head can guess a sentence boundary that the punctuation head didn't place (hello how are you → Hello, How are You?). On full passages it is rare: 0.4% of words on the official test set. Pass whole utterances, not fragments.
• Caseless-script loanwords are never capitalised (see truecasing above).
• EXCLAM is weak: 0.11 F1 on the official set and 0.07 on WIT3, and exclamation marks are rare there too.
• Only five marks. Colons, semicolons, dashes, quotes and parentheses are not restored, and French typographic spacing before ? and ! is not added.
• Domain. The benchmark is TED, which is prepared monologue. Conversational, overlapping, multi-speaker speech is harder, and isn't measured here.

Licence and provenance

Weights and code: MIT. Both base encoders are MIT-licensed. Every TTS voice used for training audio is MIT, Apache-2.0, CC0 or CC-BY-4.0.

A few things are worth knowing before commercial use:

• Wikipedia is CC-BY-SA-4.0. Whether share-alike terms reach model weights trained on such text is legally unsettled. Many encoders trained on Wikipedia, BERT among them, are released under permissive licences, and Dewpoint follows that practice.
• Tatoeba is CC-BY-2.0 FR. Sentences © Tatoeba contributors, https://tatoeba.org.
• Some Piper voices are CC-BY-4.0, which asks for attribution to the voice datasets. They are listed in results/rt2_voices.json.
• Evaluation data. IWSLT/WIT3 TED transcripts are CC-BY-NC-ND-4.0. They were used for measurement only and are not included here.

This is a description, not legal advice.

Files

The inference file reproduces the published official-set score exactly, per language, against the evaluation harness it was measured with.

Citation

@misc{dewpoint2026,
  title  = {Dewpoint: multilingual punctuation restoration and truecasing for speech},
  author = {valkayuh},
  year   = {2026},
  url    = {https://huggingface.co/valkayuh/dewpoint}
}

Please also cite the encoders Dewpoint builds on: mmBERT and XLM-RoBERTa.