Model Card for Monlam Melong preview

Model Details

Model Description

Monlam Melong is a large language model (LLM)(Tibetan LLM) specifically designed to support and enhance Natural Language Processing (NLP) tasks for the Tibetan language, a traditionally low-resource language. The model can handle a range of NLP tasks, including machine translation, named entity recognition (NER), question answering, text generation, and sentence segmentation. It supports translation from over 200 languages into Tibetan and vice versa, making it one of the most comprehensive AI models for Tibetan language processing to date.

Monlam Melong was developed as part of Monlam AI's initiative to preserve and promote the Tibetan language and cultural heritage. By building and training a Tibetan-centric LLM, MonlamMelong addresses the technological gap that has historically excluded low-resource languages from mainstream AI development.

• Developed by: Monlam AI
• Model type: Large Language Model (LLM)
• Language(s) (NLP): Tibetan, with support for multilingual translation (200+ languages)
• License: Open-Source License (details to be added)
• Finetuned from model [optional]: Custom model architecture built on top of pre-trained models

Model Sources [optional]

• Repository: [Link to model repository]
• Paper [optional]: [Link to any relevant research paper, if applicable]
• Demo [optional]: [Link to demo, if available]

Uses

Direct Use

MonlamMelong can be used directly to support a wide range of NLP tasks in the Tibetan language, including:

• Translation: Translation from Tibetan to 200+ languages and vice versa.
• Text-to-Text Generation: Writing letters, essays, or educational materials in Tibetan.
• Content Creation: Generation of Tibetan children's stories, educational content, and creative writing.
• Information Extraction: Named Entity Recognition (NER) for historical, cultural, and linguistic research.
• Text Segmentation: Sentence and paragraph segmentation for downstream NLP tasks.

Downstream Use [optional]

MonlamMelong can be fine-tuned or adapted for specialized tasks such as:

• Linguistic Analysis: Tools for researchers in linguistics or anthropology studying the Tibetan language.
• Education Apps: Use in Tibetan language learning platforms and educational tools.
• Digital Libraries: Use in text search, retrieval, and analysis for Tibetan digital archives.

Out-of-Scope Use

• Misuse for Generating Misinformation: Users should refrain from using MonlamMelong to generate false or misleading content.
• Uncontrolled Autonomy: The model should not be used in fully autonomous systems that make critical decisions without human oversight.

Bias, Risks, and Limitations

MonlamMelong inherits biases from its training data, as NLP models are often influenced by the language and perspectives present in the datasets. Special attention should be paid to the following issues:

• Linguistic Bias: Since MonlamMelong was primarily trained on Tibetan text, it may not perform as well on non-Tibetan NLP tasks.
• Cultural Representation: The model may reflect existing societal and cultural biases present in its training data, especially in sensitive or historical contexts.
• Translation Accuracy: While MonlamMelong supports translation from 200+ languages, errors may arise due to differences in sentence structure and idiomatic expressions between languages.
• Data Limitations: As a model for a low-resource language, the training data may not be as extensive as data available for high-resource languages like English or Mandarin.

Recommendations

• Human Oversight: Users should review the model's outputs, especially in educational or historical contexts, where precision and cultural sensitivity are crucial.
• Bias Audits: Institutions using MonlamMelong for translation or information extraction should regularly audit for bias in its performance.
• Fine-Tuning for Specialized Tasks: For specific academic or educational applications, fine-tuning the model may improve task-specific performance.

How to Get Started with the Model

To use Monlam Melong with the 🤗 Transformers library, you can load the model as follows:

import torch
from transformers import pipeline

pipe = pipeline(
    "text-generation",
    model="TenzinGayche/Melong_preview",
    model_kwargs={"torch_dtype": torch.bfloat16},
    device="cuda",  # replace with "mps" to run on a Mac device
)

messages = [
    {"role": "user", "content": "Please translate the following text into Tibetan: Hi how are you ? Translation: "},
]

outputs = pipe(messages, max_new_tokens=256)
assistant_response = outputs[0]["generated_text"][-1]["content"].strip()
print(assistant_response)