language:
- tum
- eng tags:
- translation
- seq2seq
- nllb
- nllb-200
- instruction-tuning
- malawi license: mit datasets:
- mwanzau/tumbuka-language-malawi metrics:
- bleu model-index:
- name: tumbuka_nllb-200 results: []
tumbuka_nllb-200 (Version 2.0)
This model is a highly optimized, fine-tuned checkpoint of Meta's facebook/nllb-200-distilled-600M, specifically adapted for high-fidelity translation and instruction-following tasks in the Tumbuka (Chitumbuka) language, spoken primarily in Northern Malawi and Eastern Zambia.
While Version 1.0 laid the groundwork for basic translation, Version 2.0 fixes deep-seated regional language leakage (such as Nyanja/Chichewa loan-words) by undergoing 9+ hours of advanced multi-stage training. It combines premium English-Tumbuka translation pairs with dedicated Tumbuka instruction-tuning datasets.
Key Improvements in V2.0:
- Linguistic Purity: Eradicated common baseline model bleed (e.g., automatically corrects baseline defaults like Takulandilani into pure Tumbuka Mwapokeleleka).
- Contextual Precision: Significantly improves technical and situational vocabulary mapping (e.g., shifting the translation of "reliable irrigation" from rigid variations to natural maji ghakugomezgeka ghakuthira).
Model Description
- Developed by: SaintsStudios
- Model type: Encoder-Decoder (Seq2Seq)
- Language(s) (NLP): English (
eng_Latn) to Tumbuka (tum_Latn), and Tumbuka Monolingual Instruction Tasking - License: MIT
- Finetuned from model: facebook/nllb-200-distilled-600M
Performance Examples
| English Input | Baseline NLLB / V1.0 Output | V2.0 Pure Tumbuka Output |
|---|---|---|
| Welcome to our home. | Takulandilani ku nyumba yithu (Nyanja leak) | Mwapokeleleka ku nyumba yithu |
| We need reliable irrigation for our farms. | Tikukhumba maji ghakukhora... | Tikukhumba maji ghakugomezgeka ghakuthira minda yithu |
| Good morning, how are you today? | Mulenji uwemi... | Mulenji uwemi, muli wuli mwahuno? |
| Who are you? | Kasi ndiwe njani? | Kasi ndiwe njani? |
How to Use the Model
You can use this model directly with the Hugging Face transformers library for inference.
Python Inference Code
import torch
from transformers import AutoTokenizer, AutoModelForSeq2SeqLM
model_id = "SaintsStudios/tumbuka_nllb-200"
print("Loading model and tokenizer...")
tokenizer = AutoTokenizer.from_pretrained("facebook/nllb-200-distilled-600M")
model = AutoModelForSeq2SeqLM.from_pretrained(model_id)
device = "cuda" if torch.cuda.is_available() else "cpu"
model.to(device)
def translate_to_tumbuka(text: str):
# Set the language configs
tokenizer.src_lang = "eng_Latn"
inputs = tokenizer(text, return_tensors="pt").to(device)
# Force target generation to start with the Tumbuka language token
forced_bos_token_id = tokenizer.convert_tokens_to_ids("tum_Latn")
translated_tokens = model.generate(
**inputs,
forced_bos_token_id=forced_bos_token_id,
max_length=128,
num_beams=4,
early_stopping=True
)
return tokenizer.decode(translated_tokens[0], skip_special_tokens=True)
# Test run
sample_text = "Let us learn together."
output = translate_to_tumbuka(sample_text)
print(f"English: {sample_text}")
print(f"Tumbuka: {output}")
# Output: Tiyeni tisambire pamoza
Training Hyperparameters (V2.0)
Dataset Blend: 50% Translation Pairs / 50% Monolingual Alpaca Instruction Datasets
- Epochs: 3
- Learning Rate: 2e-5
- Optimizer: Adafactor
- Batch Size: 2 (Gradient Accumulation Steps: 8)
- Precision: FP16 Mixed Precision
- Hardware: Kaggle Cloud Compute (GPU T4)
Acknowledgements
Special thanks to the open-source contributors maintaining datasets for Malawian localized languages, making high-quality regional adaptation pipelines possible.