Qwen3.5-4B-hsb-dsb
German to Upper Sorbian (hsb) and Lower Sorbian (dsb) translation model from the Master's
thesis "LLM-based Low-Resource Machine Translation for Sorbian Languages with Restricted
Resources" (Ahmed Eltawel, Technical University of Munich, 2026).
The model is the uniform weight average of two Qwen3.5-4B models that differ only in their training data: one was trained with 159,000 additional German-Upper Sorbian pairs produced by forward translation, the other without them.
Training
Both averaged models start from Qwen/Qwen3.5-4B and were trained following the TartuNLP recipe of the WMT25 shared task on LLMs with limited resources for Slavic languages (Purason and Fishel, 2025): continued pretraining on Sorbian monolingual text, German-Sorbian and Upper-Lower Sorbian parallel data in both translation directions and instruction data, for 2,391 steps of 128 sequences of 4,096 tokens with learning rate 1e-4, warmup, a stable phase and linear decay. Translation pairs that overlap the WMT25 development sets were removed before training. The synthetic pairs were produced by translating German News Crawl 2023 with tartuNLP/Qwen2.5-3B-Instruct-hsb-dsb.
Usage
Requires transformers 5.2 or newer.
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
model_id = "ahmedeltawel/Qwen3.5-4B-hsb-dsb"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, dtype=torch.bfloat16, device_map="auto")
language = "Upper Sorbian"
messages = [
{"role": "system", "content": f"You are a professional translator. Translate the following text from German to {language}. Answer with the translated text."},
{"role": "user", "content": "Wie lange willst du noch bleiben?"},
]
inputs = tokenizer.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt", return_dict=True).to(model.device)
output = model.generate(**inputs, max_new_tokens=256, do_sample=False)
print(tokenizer.decode(output[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True).split("\n")[0].strip())
Use language = "Lower Sorbian" for Lower Sorbian.
Training data
Sorbian parallel data and Sorbian monolingual sentences released for the WMT shared tasks on Sorbian (WITAJ-Sprachzentrum and Sorbian Institute), Upper and Lower Sorbian documents from FineWeb-2 and Wikipedia, and the instruction sets Magpie, FLAN v2, OpenAssistant 2 and Aya.
License
CC BY-NC-SA 4.0, following the license of the Sorbian training data provided by the WITAJ-Sprachzentrum. The base model Qwen/Qwen3.5-4B is licensed under Apache 2.0.
This is a research model and has not been tested for general use.