morisien-embed

A newer model exists. morisien-embed-v1.5 is fine-tuned from LaBSE instead and scores better out of domain, where this model does not beat untrained LaBSE. This checkpoint is unchanged and stays available: it is the one described in the preprint and registered in the MTEB task.

To our knowledge, the first dedicated text embedding model for Mauritian Creole (Kreol Morisien, mfe), the home language of roughly 90% of Mauritius (2022 census).

Fine-tuned from multilingual-e5-base on 35,064 Creole↔{English, French} pairs from MorisienMT and the Mauritian portion of Kreyòl-MT. On the held-out MorisienMT test split it beats every general multilingual embedding model we evaluated, including LaBSE, in all three measured retrieval directions (Creole→English, Creole→French, English→Creole).

Two corrections to earlier wording on this card. The corpus is not all publicly available Creole parallel text: google/smol carries 2,472 further gold pairs that it does not contain. And the margin over LaBSE holds in domain but not outside it, which is what the section below on FLORES+ now says.

Use it for semantic search, retrieval, RAG, bitext mining, or clustering over Kreol Morisien text.

Usage

from sentence_transformers import SentenceTransformer

model = SentenceTransformer("Singaraj/morisien-embed")

creole = ["Mo pe al bazar aster.", "Bann zanfan pe zwe dan lakour."]
english = ["I am going to the market now.", "The children are playing in the yard."]

similarity = model.similarity(model.encode(creole), model.encode(english))

Trained with Matryoshka loss, so embeddings can be truncated for faster search at a small, measured accuracy cost (ndcg@10 on the benchmark below: 0.9591 at 256 dims, 0.9531 at 128):

model = SentenceTransformer("Singaraj/morisien-embed", truncate_dim=256)

No prompt/prefix is required.

Results

Creole→English retrieval on the held-out MorisienMT test split (1,000 queries, leak-free against training data, enforced in the data pipeline by exact matching and by a punctuation-, case- and accent-insensitive check):

Creole→French, same protocol:

English→Creole, the reversed direction over the same pairs (999 queries retrieving 1,000 Creole passages, built with scripts/build_benchmark.py --reverse):

Generalization to an independent domain, FLORES+ mfe devtest (1,012 professionally translated sentences from Wikinews, Wikijunior and Wikivoyage, zero overlap with training data):

Both models sit at the ceiling of this benchmark. FLORES+ sentences are long and distinctive, so 1,012-way retrieval saturates, and this is evidence of no out-of-domain degradation rather than a margin over LaBSE.

That reading was still too generous to the pool. A character n-gram TF-IDF baseline with no neural model scores 0.8221 accuracy@1 on it, so retrieving the right passage among 1,012 needs little more than surface overlap. On a harder pool built from the same sentences with xSIM++ distractors, this model does not beat untrained LaBSE. See the preprint and morisien-embed-v1.5.

The contrastive stage was repeated with three random seeds over the same deterministically mined negative set; Creole→English test ndcg@10 across seeds: 0.9653 ± 0.0002 (accuracy@1 0.9433 ± 0.0006). The released checkpoint is seed 42, designated before results were seen.

Every number in the tables above is reproducible from the training repository (Matryoshka figures via scripts/evaluate.py --truncate-dim). The Haitian-proximity figures under Limitations come from scripts/probe_haitian.py.

MTEB

The held-out MorisienMT test split is now a task in MTEB, MorisienMTBitextMining, the first Mauritian Creole task in the benchmark. This model is registered in MTEB and its scores are on the leaderboard.

Bitext-mining F1 across the four directional subsets:

This is bitext-mining F1, a different metric from the ndcg@10 retrieval numbers above. The model is trained on the MorisienMT corpus this split is drawn from, so MTEB records the result as in-domain (via training_datasets), not zero-shot.

Training

• Data: 35,064 unique, leak-free Creole↔{English, French} pairs, merged from MorisienMT (MIT) and Kreyòl-MT (mixed licenses; used for training only, not redistributed). Every MorisienMT dev/test sentence is checked against training by exact matching and by a punctuation-, case- and accent-insensitive check. Both are checks rather than removal steps: they drop 0 of the 69,525 raw rows, because the evaluation splits and the training sources are already disjoint. Verifiably absent is a stronger claim than removed, and it is the one the measurement supports.
• Recipe: hard-negative mining with positive-aware false-negative filtering (mine_hard_negatives: 5 negatives/anchor, range_min=10, relative_margin=0.05). The margin filter is strict: 24,100 of the 35,064 pairs survived with a full negative set, and the released checkpoint's contrastive stage trained on those 24,100 tuples (the stage-1 mining model itself was trained on all 35,064). Contrastive training uses CachedMultipleNegativesRankingLoss (batch 128, 767 in-batch negatives per anchor) wrapped in MatryoshkaLoss (dims 768/512/256/128/64). 3 epochs, lr 2e-5, warmup 10%, fp16, seed 42, single T4 GPU (~30 min contrastive + ~11 min mining).
• Base model: intfloat/multilingual-e5-base (278M parameters, MIT).

Limitations

• Not native-perfect. Accuracy@1 around 0.944 means roughly one query in eighteen ranks a wrong translation first. Strong, but below a human bilingual speaker.
• Register. MorisienMT's authors built it from books translated from English, the Bible among them, plus basic sentences written by hand (Dabre and Sukhoo 2022, §4). Informal and technical text are untested.
• Small evaluation universe. Retrieval is measured over ~1,000-passage corpora, standard for bitext benchmarks, but absolute scores would be lower against web-scale corpora.
• The margin over LaBSE is in-domain only. MorisienMT and Kreyòl-MT overlap heavily. On the harder xSIM++ pool built from FLORES+, this model does not beat untrained LaBSE; see the preprint and morisien-embed-v1.5.
• Haitian Creole proximity. Like every multilingual embedder we tested, the model embeds Haitian Creole close to Mauritian Creole. Measured on the 1,012 aligned mfe/hat/eng FLORES+ devtest triplets (scripts/probe_haitian.py): with every same-meaning Haitian twin injected into the corpus, mfe→eng accuracy@1 drops from 1.00 to 0.68, and LaBSE resists this trap better (0.79). Asked instead to tell the two creoles apart (is the English sentence closer to its Mauritian or its Haitian translation?), the fine-tune picks Mauritian 709/1012 times vs LaBSE's 351/1012. Wrong-meaning Haitian text is never confused; mixed mfe/hat corpora will still degrade retrieval.
• Not measured here: pure-English quality after fine-tuning, and behaviour on ALL-CAPS text. Use a general model for English-only workloads; this model is for Creole and Creole↔{English, French} work.
• Long inputs are truncated at the encoder's maximum sequence length; chunk long documents before embedding.
• Protocol note. During recipe development the held-out test score was printed at the end of each training run, so recipe selection had test visibility. An earlier version of this card bounded the resulting optimism at about 0.01 ndcg; that bound is withdrawn, because no committed script produces it. The 3-seed replication was run after the recipe was frozen. Leak filtering reserves the Creole side of every evaluation pair; English/French target texts are not reserved, so a benchmark passage can also occur in training as the translation of a different Creole sentence. The accent-insensitive half of the leak check was added after the released run; it verifiably leaves the training set byte-identical, since the sources were already disjoint at that level.
• Orthographic variation. Training data mixes pre- and post-2011 (Lortograf Kreol Morisien) spellings; performance on older orthography is untested.

Citation

If you use this model, please cite the accompanying report along with the data sources it builds on: MorisienMT (Dabre & Sukhoo, 2022) and Kreyòl-MT (Robinson et al., NAACL 2024).

@misc{morisien-embed,
  author    = {Singaraj B},
  title     = {morisien-embed: A Dedicated Text Embedding Model and Benchmark for Mauritian Creole (Kreol Morisien)},
  year      = {2026},
  publisher = {Zenodo},
  doi       = {10.5281/zenodo.21877805},
  url       = {https://doi.org/10.5281/zenodo.21877805}
}