Homo-GE2PE-Persian-HF

This is not a new model. It is a repackaging of MahtaFetrat/Homo-GE2PE-Persian by Elnaz Rahmati et al. — the weights are byte-identical. All credit for the model belongs to the original authors.

The -HF suffix follows the Llama-2-7b-hf convention: same weights, packaged in transformers-native format rather than the original release format. It is not a new version of the model.

What changed is only the packaging: the original ships zipped weights plus a vendored copy of Parsivar and a custom GE2PE.py wrapper. This repo is a plain from_pretrained() model with no extra dependencies, so it can be dropped into an inference pipeline directly.

Repackaged for use as the pronunciation frontend of pocket-tts-farsi, a Persian TTS model that runs on CPU.

Why drop Parsivar

Parsivar was used only to normalize input text before the model. Measured on SentenceBench (400 sentences), removing it improves every metric, because it merges prepositions into the following word (beqadre where the reference has be qadar-e):

Any reasonable Persian normalizer works in its place — or none at all on already-clean text. Do use one in production, though: real input contains Arabic ي/ك, stray digits, and ZWNJ inconsistencies that the model has not been trained to handle.

Usage

import torch
from transformers import AutoTokenizer, T5ForConditionalGeneration

REPO = "mehdi-hf/Homo-GE2PE-Persian-HF"
tok = AutoTokenizer.from_pretrained(REPO)
model = T5ForConditionalGeneration.from_pretrained(REPO).eval()

def g2p(sentences: list[str]) -> list[str]:
    # add_special_tokens=False and 5 beams match how the model was trained.
    enc = tok(sentences, padding=True, add_special_tokens=False,
              return_attention_mask=True, return_tensors="pt")
    with torch.no_grad():
        out = model.generate(enc["input_ids"], attention_mask=enc["attention_mask"],
                             num_beams=5, min_length=1, max_length=512,
                             early_stopping=True)
    return [s.strip() for s in tok.batch_decode(out, skip_special_tokens=True)]

print(g2p(["حملات برون‌مرزی", "من قدر تو را می‌دانم"]))
# ['h/m/late1 borunm/rzi', 'm/n q/dre1 to ra midan/m']

Output notation — read this before comparing to anything

The model emits GE2PE's internal romanization, which differs from the notation used in SentenceBench's phoneme column even though both come from the same authors. Comparing raw strings without mapping scores PER ~30% instead of ~3%.

_TO_REF = str.maketrans({"/": "a", "a": "A", "@": "?", "$": "S", "c": "C"})

def to_conventional(text: str) -> str:
    return text.translate(_TO_REF).replace("1", "")

to_conventional("h/m/late1 borunm/rzi")   # 'hamalAte borunmarzi'

The 1 marks where an ezafe was inserted. The original wrapper strips it via .replace('1', ''), but only on sentences whose word counts happen to align, so strip it unconditionally.

What it does well

Persian script omits short vowels and never writes the ezafe (the linking -e between a noun and its modifier), so pronunciation must be inferred:

That last one is homograph disambiguation, which is the original paper's main contribution.

Credit

The model, the training data, and the research are entirely the work of the original authors:

• Model: MahtaFetrat/Homo-GE2PE-Persian
• Dataset: HomoRich — 528,891 annotated Persian sentences
• Benchmark: SentenceBench
• Paper: Fast, Not Fancy: Rethinking G2P with Rich Data and Rule-Based Models

@article{rahmati2025fast,
  title={Fast, Not Fancy: Rethinking G2P with Rich Data and Rule-Based Models},
  author={Rahmati, Elnaz and others},
  journal={arXiv preprint arXiv:2505.12973},
  year={2025}
}

License

MIT, © 2025 Elnaz Rahmati — see LICENSE. Base model google/byt5-small is Apache 2.0, © Google LLC — see LICENSE-apache-2.0.md. Full attribution in NOTICE.md.