ByT5 multilingual G2P (small) — 300M params
Byte-level seq2seq, 142 language/ variety tags (eng-US/eng-UK, por-BR/por-PT, spa-ES/spa-LatAm, Welsh N/S, Armenian E/W, Bengali varieties, 20+ Sinitic splits). Trained on a 4.12M-pair harmonized corpus (+27% over the previous 3.02M), with IPA conventions unified per language via a learned M2M aligner.
Input: <lang>: word (ISO-639-3, variety-suffixed where split
— <eng-US>: hello, <spa-ES>: abeja). Output: space-separated IPA
in gruut-anchored convention (h ɛ l ˈoʊ) — what Piper-family
voices consume directly.
Results (4k stratified test sample, same test set)
| this model | previous (0.731 on old test) | |
|---|---|---|
| micro exact | 0.730 | 0.663 (same test, same eval) |
| macro exact | 0.617 | 0.620 |
The new model is trained on a 30% larger, harmonized corpus where source conventions (gruut + WikiPron) are aligned per language with a many-to-many EM aligner. On the same held-out test set, micro exact climbs +6.7 pts (0.663 → 0.730). The old published 0.731 was on a different test set with mixed conventions — the new 0.730 is on a harder test (one convention per language, no ties to exploit).
Highlight wins: Spanish 0.720 → 0.976, French 0.694 → 0.917, Portuguese 0.000 → 0.570 (was WikiPron-only, now gruut-anchored), Dutch 0.014 → 0.188.
Files
- HF-format weights at root (~1.2 GB)
onnx/— validated encoder+decoder pair (manual TorchScript export, 3/3 gate-pair validation passing). Consume withonnx_reference.py(a minimal correct consumer). CRITICAL conventions: token id = byte + 3; EOS appended to encoder input; decoder needs an explicit causal mask and a length-2 bootstrap — the reference script encodes all of them.
Quickstart (ONNX, CPU)
# pip install onnxruntime numpy
from huggingface_hub import snapshot_download
import sys; sys.path.insert(0, "voicegarden-lexicons/scripts/train_byt5")
from onnx_reference import load, run
d = snapshot_download("willwade/byt5-g2p-multilingual", allow_patterns=["onnx/*"])
enc, dec = load(d + "/onnx")
print(run(enc, dec, "<eng-US>: floravox"))
Training
The checkpoint was retrained on a 4.12M-pair harmonized corpus built
from gruut + WikiPron sources with M2M-convention alignment. Training
details in the
voicegarden-lexicons
repository (scripts/train_byt5/). Run: RTX 3090, ~8h, $1.20.
Licence
CC BY-SA 4.0 (share-alike inherited from WikiPron training data). Attribution: Wiktionary/WikiPron (CUNY-CL), gruut (rhasspy), Google byt5 base (Apache-2.0). Training code: voicegarden-lexicons/scripts/train_byt5.