Model Card for Model ID

Due to compute issues, The model has been trained on multiple iterations:

1. Model Trained for 8500 steps on [0 : 5%] of the dataset.
2. Model continued from 8500 upto 16500 steps on [5% : 20%] of the dataset
3. Model continued from 16500 upto 22000 steps on [20% : 40%] of the dataset

Model Description

This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated.

• Model type: (Translation)
• Language(s) (NLP): Nepali, English
• License: [Apache license 2.0]
• Finetuned from model : [google/mt5-small]

How to Get Started with the Model

Use the code below to get started with the model.

from transformers import AutoTokenizer, MT5ForConditionalGeneration

checkpoint = "syubraj/RomanEng2Nep-v2"
tokenizer = AutoTokenizer.from_pretrained(checkpoint)
model = MT5ForConditionalGeneration.from_pretrained(checkpoint)

# Set max sequence length
max_seq_len = 20

def translate(text):
    # Tokenize the input text with a max length of 20
    inputs = tokenizer(text, return_tensors="pt", truncation=True, max_length=max_seq_len)

    # Generate translation
    translated = model.generate(**inputs)

    # Decode the translated tokens back to text
    translated_text = tokenizer.decode(translated[0], skip_special_tokens=True)
    return translated_text

# Example usage
source_text = "muskuraudai"  # Example Romanized Nepali text
translated_text = translate(source_text)
print(f"Translated Text: {translated_text}")

Training Data

syubraj/roman2nepali-transliteration

Training Hyperparameters

• Training regime:

training_args = Seq2SeqTrainingArguments(
    output_dir="/content/drive/MyDrive/romaneng2nep_v2/",
    eval_strategy="steps",
    learning_rate=2e-5,
    per_device_train_batch_size=16,
    per_device_eval_batch_size=8,
    weight_decay=0.01,
    save_total_limit=3,
    num_train_epochs=2,
    predict_with_generate=True,
)

Training and Validation Metrics