mitra-qwen35-base-stage2
The 9B foundation model of the Dharmamitra stack:
Qwen3.5-9B after large-scale domain adaptation to classical Buddhist
literature. Stage 2 adds translation-specific capabilities on top of the
stage-1 pretraining: bidirectional parallel translation training
(Sanskrit↔English, Sanskrit↔German, and related classical pairs) plus full
production translation/research turnarounds at 16k context. This is the model that
buddhist-nlp/mitra-qwen35-embedder
was finetuned from, released for research use as a starting point for
Buddhist-NLP downstream tasks (translation, retrieval finetunes, QA, philology
tooling).
Training
Two stages on top of Qwen3.5-9B:
- Continued pretraining (~30B tokens, 8k context) on classical Buddhist corpora: Sanskrit, Tibetan (Wylie transliteration), Buddhist Chinese, and Pāli source texts with related secondary literature.
- Stage-2 finetuning (16k context, 1 epoch) on a mixture of bidirectional parallel translation data (Sanskrit↔English, Sanskrit↔German, and related pairs), full production translation/research turnarounds, monolingual replay, and general instructions.
The result is an instruction-following model with strong classical-language competence in reading, translating, and discussing Buddhist source texts.
Conventions
- Tibetan input/output is Wylie transliteration, not Tibetan script.
- Sanskrit and Pāli are IAST romanization.
- A chat template is bundled (
tokenizer.apply_chat_template).
Usage
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
tok = AutoTokenizer.from_pretrained("buddhist-nlp/mitra-qwen35-base-stage2")
model = AutoModelForCausalLM.from_pretrained(
"buddhist-nlp/mitra-qwen35-base-stage2", dtype=torch.bfloat16, device_map="cuda"
)
messages = [{"role": "user", "content":
"Translate into English: evaṃ mayā śrutam ekasmin samaye"}]
inputs = tok.apply_chat_template(messages, add_generation_prompt=True,
return_dict=True, return_tensors="pt").to(model.device)
out = model.generate(**inputs, max_new_tokens=128)
print(tok.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))
Relation to other releases
buddhist-nlp/mitra-qwen35-embedder— retrieval embedder finetuned from this model (state of the art on cross-lingual Buddhist retrieval).
Citation
If you use this model, please cite the Dharmamitra project (https://dharmamitra.org). A technical report is in preparation.