Punctuation Restoration (base, 307M): 88 languages

Adds punctuation back to text that has none: speech-recognition (ASR) transcripts, subtitles, chat messages, voice notes, OCR or scraped text. It predicts, after each word, one of . , ? : - or nothing, in 88 languages, with any casing (including the all-lowercase output of many ASR systems). Built on mmBERT-base, Apache-2.0, ONNX files for CPU and the browser (transformers.js) included. Try it in the browser. A smaller, faster version is available as punctuation-restoration-small.

• Same labels as oliverguhr/fullstop-punctuation-multilang-large and kredor/punctuate-all (0 . , ? - :), so it is a drop-in for the deepmultilingualpunctuation package.
• Chinese and Japanese work too (no word segmentation needed) with the included punctuate.py, which writes ，。？： there, and । (Hindi, Bengali), ، ؟ (Arabic script), ; for questions in Greek and ። ፣ in Amharic.
• It restores punctuation only. It does not change casing or the words.
• onnx/model_quantized.onnx (int8 embeddings, 641 MB) predicts the same label as fp32 for 99.93% of tokens on held-out text.

Usage

With the included helper (any language; long texts are processed in overlapping chunks):

from huggingface_hub import hf_hub_download
import importlib.util, sys
spec = importlib.util.spec_from_file_location("punctuate", hf_hub_download("Horizon-Labs/punctuation-restoration-base", "punctuate.py"))
punctuate = importlib.util.module_from_spec(spec); spec.loader.exec_module(punctuate)

p = punctuate.Punctuator("Horizon-Labs/punctuation-restoration-base")
print(p("hello how are you today i hope everything is fine"))
print(p("今天天气很好我们去公园散步吧你觉得怎么样"))

As a drop-in for deepmultilingualpunctuation (space-separated languages; that package needs transformers 4.x, because it passes grouped_entities, which transformers 5 removed):

from deepmultilingualpunctuation import PunctuationModel
model = PunctuationModel(model="Horizon-Labs/punctuation-restoration-base")
print(model.restore_punctuation("my name is anna i live in berlin what about you"))

Plain transformers: the label of each word is the label of its last token ("0" = no mark).

from transformers import pipeline
tagger = pipeline("token-classification", model="Horizon-Labs/punctuation-restoration-base")
tagger("hello how are you today")

transformers.js:

import { pipeline } from "@huggingface/transformers";
const tagger = await pipeline("token-classification", "Horizon-Labs/punctuation-restoration-base", { dtype: "q8" });
console.log(await tagger("hello how are you today i hope everything is fine"));

Evaluation

Punctuation F1: micro-averaged over the five marks (a predicted mark counts only if it is the correct mark at the correct position), averaged over languages. Every model gets the same input: the paragraph with . , ? : ; ! - removed, once with its original casing and once lowercased (as most ASR output is). Same script for every model (code/). The test sets were used only for evaluation:

• FLORES-200 devtest: professionally translated news, travel and wiki text; the consecutive sentences of one article form a paragraph (about 280 paragraphs per language). Columns: the 12 languages of punctuate-all, and our 88 training languages.
• TED talks (TED2020 via OPUS): transcripts and their translations, 5 consecutive sentences, 200 paragraphs in each of 25 languages. Spoken style.
• Europarl: European Parliament proceedings, 5 consecutive sentences, 200 paragraphs in each of 12 languages. punctuate-all and the fullstop models were trained on Europarl, so this column is in-domain for them; we never trained on it.

• The 1-800-BAD-CODE models expect lowercased input and also restore casing and split sentences; they are scored only on lowercased input, through the punctuators package, by re-reading their output against the input words. Words their output changed beyond casing (about 2% of words for xlm-roberta_punctuation_fullstop_truecase, 11% for punct_cap_seg_47_language) could not be aligned and count as "no mark", which lowers their scores somewhat.
• On Europarl, punctuate-all (trained on it) is ahead of this model, most clearly on lowercased input.

Per mark (this model; mean over languages, FLORES over our 88):

Training

• Data: 1,168,626 text windows (119M words) in 88 languages from FineWeb-2 and FineWeb (ODC-BY). The labels come from the text's own punctuation: snippets that end in a full stop, are mostly letters and are not lists; 1-4 snippets per window (up to 230 words), 30% cut mid-sentence. 70% of the inputs were lowercased.
• Spoken-style text (v1.1): 49,144 windows of edited transcripts of everyday speech (talks, podcasts, interviews, meetings, lectures, vlogs, phone calls ...) in 69 languages, written by Qwen3.8-27B (Apache-2.0) for this purpose; transcripts that shared a sentence with any test set were removed.
• Normalisation: !, ;, … and script full stops (。 । ። ۔ ...) count as .; ，、، as ,; ？؟ (and ; in Greek) as ?; a free-standing dash as -. A . before a lowercase word (abbreviations) counts as no mark.
• Model: mmBERT-base token classification, one label per word on its last token, 1 epoch, learning rate 3e-5. Checkpoint chosen on held-out web text, not on the test sets.
• Code: code/ in this repository.

Changelog

• v1.1 (2026-10-09): + 49k windows of synthetic spoken-style transcripts (69 languages). TED talks (spoken, 25 languages), two training seeds: punctuation F1 .784 -> .796 (cased) and .707 -> .723 (lowercased); : F1 .544 -> .605. FLORES (written text, 88 languages) unchanged (.847 / .804).
• v1.0 (2026-10-03): first release.

Limitations

• Only . , ? : - are predicted. Exclamation marks, semicolons, quotes, brackets and Spanish inverted marks are not restored. : and especially - (a free-standing dash) are much less reliable than . , ? (see the per-mark table).
• Accuracy on lowercased text is lower than on cased text, because capital letters reveal sentence starts.
• Trained mostly on written web text, plus synthetic spoken-style transcripts. Disfluent speech (fillers, false starts, repetitions) is harder, and punctuation conventions vary by writer. Thai, Lao, Khmer, Burmese and Tibetan are not supported.
• Low-resource languages and languages outside the 88 score lower (see the per-language table).