Qwen2.5-1.5B-Instruct-hsb-dsb
German to Upper Sorbian (hsb) and Lower Sorbian (dsb) translation model from the Master's
thesis "LLM-based Low-Resource Machine Translation for Sorbian Languages with Restricted
Resources" (Ahmed Eltawel, Technical University of Munich, 2026).
The model was trained with a re-implementation of the NRC submission to the WMT25 shared task on LLMs with limited resources for Slavic languages (Larkin et al., 2025): full supervised fine-tuning of Qwen2.5-1.5B-Instruct with LLaMA-Factory on the German-Sorbian and Upper-Lower Sorbian parallel data released for the WMT shared tasks on Sorbian (2020-2022 and 2025), with the WMT25 development pairs removed (855,350 training examples, 20 epochs, learning rate 7e-5, cosine schedule, batch size 256).
Usage
The model answers in the tagged format it was trained on.
import re
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
model_id = "ahmedeltawel/Qwen2.5-1.5B-Instruct-hsb-dsb"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, dtype=torch.bfloat16, device_map="auto")
code, language = "hsb", "Upper Sorbian"
text = "Wie lange willst du noch bleiben?"
messages = [
{"role": "user", "content": f"Translate the following German text to {language}. Put it in this format <{code}> {language} translation </{code}>.\n<deu> {text} </deu>"},
]
inputs = tokenizer.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt", return_dict=True).to(model.device)
output = model.generate(**inputs, max_new_tokens=256, do_sample=False)
answer = tokenizer.decode(output[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True)
match = re.search(rf"<{code}>\s*(.+)", answer.split(f"</{code}>")[0])
print(match.group(1).strip() if match else answer)
Use code, language = "dsb", "Lower Sorbian" for Lower Sorbian.
License
CC BY-NC-SA 4.0, following the license of the Sorbian training data provided by the WITAJ-Sprachzentrum. The base model Qwen2.5-1.5B-Instruct is licensed under Apache 2.0.
This is a research model and has not been tested for general use.