ebible_m2m-ie-big-shareable

A multilingual, from scratch encoder decoder transformer that translates verses of the Bible from a fixed Koine Greek source into many languages. It reproduces and extends the closed text translation experiment first carried out by Sami Liedes in 2018, replacing his convolutional sequence to sequence model with a modern transformer and adding quantitative evaluation. This checkpoint comes from the ie_big_shareable experiment.

The model is trained on verse aligned Bible translations from the eBible corpus. Whole books are withheld from certain languages during training, and the model then generates those withheld books. The practical aim is to draft scripture in a language for which parts of the Bible do not yet exist, using the many translations the model has already seen as context.

How to use

from transformers import MarianMTModel, MarianTokenizer

model = MarianMTModel.from_pretrained("DavidCBaines/ebible_m2m-ie-big-shareable")
tokenizer = MarianTokenizer.from_pretrained("DavidCBaines/ebible_m2m-ie-big-shareable")

# Prepend the target language tag (for example <2spa> for Spanish) to the
# Koine Greek source verse.
source = "<2spa> Ἐν ἀρχῇ ἐποίησεν ὁ Θεὸς τὸν οὐρανὸν καὶ τὴν γῆν"
batch = tokenizer([source], return_tensors="pt")
generated = model.generate(**batch, num_beams=5, max_length=192)
print(tokenizer.decode(generated[0], skip_special_tokens=True))

Model details

• Architecture: MarianMT style encoder decoder, randomly initialised (no pretrained weights).
• Parameters: approximately 210 million.
• Source language: composite Koine Greek (Brenton Septuagint for the Old Testament, Tischendorf for the New Testament), in Greek script.
• Target languages: indicated by an atomic <2xxx> tag prepended to the source verse.
• Tokeniser: SentencePiece, trained on the training split only.

Held out books

The following books were withheld from training and generated by the model.

Evaluation

Scores are corpus level per held out book, following the silnlp and sacreBLEU conventions. chrF3 is the headline metric. Two baselines are reported for context. The source copy baseline (copy_chrF3) returns the Greek source verse unchanged, and sits near zero because the source is in a different script. The other language baseline (other_chrF3, with the language in other_lang) is the score of the single most similar training language's own text for the same verses, which is a far more demanding floor because a close relative shares script and vocabulary with the target.

Scores are verse-weighted across each language's held-out books. Per-book metrics for every book are in generated/metrics.csv.

Training data and licensing

This model was trained only on translations whose licences permit derivative works, so that the model may itself be shared. The published model is released under cc-by-sa-4.0. Every source translation and its licence is listed below.

Reproducibility

• Experiment: ie_big_shareable
• Git commit: ef0c7ccec0d79f213e93ceb4426e22398676995b
• Random seed: 13

The full training and evaluation code, configuration and selection list are in the project repository.

Acknowledgement

This work reproduces the closed text Bible translation experiment described by Sami Liedes in his 2018 blog post "Machine translating the Bible into new languages". The reproduction and this model are independent work and are not endorsed by him.