Haidass-Translate-143M-Instruction
English | 中文
The instruction-tolerant sibling of DALabCommunity/Haidass-Translate-143M: same 143M zh⇄en translation training, plus 9.1% cleaned general-domain data (STEP_FUN ShareGPT) mixed in. Translation scores are within 0.3 BLEU of the pure-translation version, and the model retains limited general instruction-following ability that the pure-translation version does not have.
Live demo: Haidass Translate Instruction on Hugging Face Spaces.
FLORES-200 dev
| Model | Params | Arch | en→zh BLEU | en→zh chrF++ | zh→en BLEU | zh→en chrF++ |
|---|---|---|---|---|---|---|
| HY-MT1.5-1.8B | 1800M | LLM | 44.65 | 30.98 | 27.68 | 57.96 |
| Qwen3-0.6B | 600M | LLM | 30.94 | 21.10 | 20.21 | 48.62 |
| OPUS-MT en-zh | 78M | Seq2Seq | 30.88 | 21.80 | - | - |
| OPUS-MT zh-en | 78M | Seq2Seq | - | - | 22.99 | 51.03 |
| Qwen2.5-0.5B-Instruct | 500M | LLM | 28.96 | 19.65 | 18.09 | 45.85 |
| M2M-100-418M | 418M | Seq2Seq | 28.04 | 20.53 | 20.58 | 48.79 |
| Haidass-Translate-143M | 143M | LLM | 23.89 | 17.55 | 14.65 | 40.76 |
| Haidass-Translate-143M-Instruction | 143M | LLM | 23.64 | 17.21 | 14.35 | 40.13 |
| NLLB-200-distilled-600M | 600M | Seq2Seq | 22.44 | 16.74 | 25.71 | 52.28 |
| Drafter-143M* | 143M | LLM | 12.04 | 9.43 | 5.47 | 27.31 |
*Drafter-143M: a control model with identical configuration, data and training recipe, except that it starts from random initialization instead of the pretrained base — used to quantify the contribution of base-model pretraining.
OPUS-MT models are single-directional — one independent 78M model per direction; "-" marks directions a model does not serve.
FLORES+ devtest
The same models re-evaluated on FLORES+ devtest (released 2026; zero overlap with dev):
| Model | Params | Arch | en→zh BLEU | en→zh chrF++ | zh→en BLEU | zh→en chrF++ |
|---|---|---|---|---|---|---|
| HY-MT1.5-1.8B | 1800M | LLM | 37.36 | 26.08 | 20.33 | 51.48 |
| OPUS-MT en-zh | 78M | Seq2Seq | 32.23 | 22.40 | - | - |
| OPUS-MT zh-en | 78M | Seq2Seq | - | - | 23.06 | 51.03 |
| Qwen3-0.6B | 600M | LLM | 31.76 | 21.48 | 19.66 | 48.14 |
| Qwen2.5-0.5B-Instruct | 500M | LLM | 29.32 | 19.95 | 18.04 | 46.00 |
| M2M-100-418M | 418M | Seq2Seq | 28.29 | 20.60 | 19.52 | 47.87 |
| Haidass-Translate-143M | 143M | LLM | 25.06 | 17.72 | 13.68 | 40.02 |
| Haidass-Translate-143M-Instruction | 143M | LLM | 24.74 | 17.77 | 13.95 | 39.93 |
| NLLB-200-distilled-600M | 600M | Seq2Seq | 23.07 | 16.94 | 24.30 | 51.48 |
| Drafter-143M* | 143M | LLM | 10.93 | 9.00 | 5.82 | 26.83 |
Instruction following
Mixing 9.1% general-domain data into the translation SFT gives this variant a limited ability to respond to general instructions (short poems, simple summaries, paraphrases) that the pure-translation sibling essentially lacks. This ability is fragile: prompts outside the translation domain may produce echoes of the input, partial answers, or drifting continuations. The model is still primarily a translator; treat general instruction following as a bonus, not a guarantee.
Training recipe
- Base: Haidass1.5-143M (Qwen3 architecture: 30 layers, hidden 576, GQA 9/3, vocab 64,000)
- Data: 7.837M cleaned zh↔en parallel sentence pairs (15.67M samples bidirectional) + 157,593 cleaned STEP_FUN ShareGPT conversations (9.1% of training tokens)
- Packing: official MindSpeed-LLM
--pack --neat-pack(683,227 full 2048-token sequences with inter-document attention-mask isolation) - Training: 16×Ascend 910C, GBS=256, lr 3e-5 cosine, 2 epochs = 5,337 steps (~2.8B tokens), final loss 1.788
- Framework: MindSpeed-LLM v2.3.0 + Megatron-LM core_v0.12.1 (NPU)
Usage
from transformers import AutoModelForCausalLM, AutoTokenizer
model = AutoModelForCausalLM.from_pretrained("umeiko/Haidass-Translate-143M-Instruction", torch_dtype="bfloat16", device_map="auto")
tok = AutoTokenizer.from_pretrained("umeiko/Haidass-Translate-143M-Instruction")
msgs = [{"role": "user", "content": "将以下文本翻译为英文：光子甚至比构成原子的物质还要小！"}]
ids = tok.apply_chat_template(msgs, add_generation_prompt=True, return_tensors="pt").to(model.device)
out = model.generate(ids, max_new_tokens=512, do_sample=False)
print(tok.decode(out[0][ids.shape[1]:], skip_special_tokens=True))
# Even photons are smaller than the stuff that makes up atoms!
Note: the training data follows the qwen3 chat template (an empty <think></think> block precedes the assistant turn). Always use the model's built-in chat_template at inference; do not hand-craft prompts.
Known limitations
- zh→en BLEU plateaus at ~14 across data scales; more same-distribution parallel data does not push this direction further
- Typical residual errors: entity mix-ups, occasional omission of numeric details; rare-entity-dense zh→en inputs can trigger repetitive looping under greedy decoding (use
repetition_penalty/no_repeat_ngram_sizeas mitigation) - General instruction following is limited and unstable (see above)
- Optimized for zh⇄en; not a general chat model
Evaluation
- Metrics: sacreBLEU corpus BLEU (
tokenize=zhfor Chinese targets,tokenize=13afor English) + chrF++ (word_order=2); prompts byte-identical to the training chat template - Decontamination: all 15.83M training samples audited against both benchmarks by n-gram overlap (8-word English / 10-character Chinese fragments): 1,147 hits (0.0072%) vs FLORES-200 dev, 1,788 (0.0113%) vs FLORES+ devtest; the STEP_FUN-only subset shows 6 hits (0.0038%) vs devtest. Manual inspection shows common-phrase-level overlap, not full-sentence leakage
- Raw predictions, audit reports and evaluation scripts for every model in the tables above: umeiko/Haidass-Translate-143M-eval
Note: all scores are measured on the FLORES Chinese–English subset (eng_Latn ↔ zho_Hans), bidirectional (997 sentences for dev, 1,012 for devtest), with greedy decoding.