Moxhi MT 30 QT (zh-vi)

QT/convert-register variant of MoxhiMT-30: familiar QT-style pronouns (ta/ngươi/hắn/nàng/tỷ muội) with neural translation quality — and rock-stable pronoun register across a whole chapter.

Standard NMT models (including MoxhiMT-30) translate sentence-by-sentence and drift between pronoun registers mid-chapter: ta ↔ tôi, chúng ta ↔ chúng tôi, tỷ muội ↔ chị em. This variant was trained on the same corpus with all targets normalized to a single QT register, so the instability is gone at the source.

What's new in v1.1 (2026-09-21)

No more guessed kinship pronouns in dialogue. Whenever a line carried a vocative (师姐 / 哥 / 姐 / 娘 / 爹 …), v1.0 tried to work out how the two speakers were related and rendered 我 / 你 as kinship or gendered pronouns — em, muội, chàng, anh, con. A single sentence rarely says who is older or which gender is speaking, so the guess was often wrong: a male speaker became muội, and a daughter asking her mother "你当年为何离开宗门？" got con for 你, which turns the question onto the wrong person. v1.1 follows the source character instead: 我 → ta, 你 → ngươi, 您 → ngài. Kinship words appear only where the source has them (师姐 → sư tỷ, 妹妹 → muội muội).

Outputs above are reproducible with the Quick Start below (num_beams=4, plus the post-processing step). The last row is the second change in v1.1: the word right after an opening quote used to be spelled out piece by piece, which broke rare names in that position.

Measured on 1,200 held-out dialogue lines (novel-level split, hash-gated against the training pool; 1,226 quoted regions) and the 1,392-row clean suite — CT2 int8, greedy, repetition_penalty=1.05:

Trade-off. Two name probes are lost (卢耀 → Lư Diệu, the standard reading, where the reference says Lô; 周青 → a wrong third syllable in both versions, scored as a hit for v1.0 only by substring). A 37M model forgets mid-frequency names quickly, which is why v1.1 is a deliberately small update: the 8k-step "opener-spacing" continuation of v1.0 plus 600 SFT updates at LR 1e-5 on 6.5k teacher-translated dialogue lines and 30k clean replay rows.

Output convention — one post-processing step (new in v1.1)

v1.1 emits one extra space after an opening quote or bracket (“ Sư tỷ, ta đi…”). This is intentional: it lets the first word of a quote use the same well-trained token as everywhere else. Remove that space after decoding:

import re

_OPENERS = "“‘«「『【《([{（［｛"
_OPENER_SPACE = re.compile("([" + re.escape(_OPENERS) + r"]) (?=[^\W\d_])")

def postprocess(text: str) -> str:
    """Drop the single synthetic space between an opening mark and a letter."""
    return _OPENER_SPACE.sub(r"\1", text)

JavaScript: text.replace(/([“‘«「『【《([{（［｛]) (?=\p{L})/gu, "$1"). Skipping the step only leaves a cosmetic space; the translation is unaffected. v1.1 also always writes curly quotes “ ” (v1.0 sometimes copied straight ones).

Same architecture, tokenizer and config as v1.0 — a drop-in update apart from the step above. Pin a release with revision=: v1.1 (this one), v1.0.3 (v1.0 weights + the current tokenizer), v1.0 (original upload).

Measured consistency (7 chapters, 538 lines, multi-genre)

Flip rate = register switches between consecutive lines of the same class (table measured on v1.0; v1.1 brings the 2nd-person row to 0.000 as well — see above). Overall translation quality is on par with the same-recipe baseline (clean-suite BLEU 50.3 vs 50.4 register-normalized refs; blind human-arbitrated review: 26 wins / 16 losses / 9 ties on 60 lines).

Trade-offs (by design)

• Global QT voice: modern-setting prose also uses ta/ngươi/hắn — the convert-literature convention. If you want natural modern-Vietnamese register, use MoxhiMT-30 instead.
• 我们/咱们 → chúng ta always: the inclusive/exclusive distinction (chúng ta/chúng tôi) is intentionally flattened, following QT convention.
• 你 → ngươi even towards elders (v1.1): "Nương, sao ngươi lại khóc?" is the convert-literature convention, not natural spoken Vietnamese. The model no longer guesses relationships it cannot see.
• No DPO pass yet (this is the base one-knob variant).

Model Details

• Architecture: Marian seq2seq (asymmetric 8 encoder + 2 decoder) — same as MoxhiMT-30; drop-in interchangeable (same tokenizer/config), different voice.
• Parameters: ~37M
• Tokenizer: SentencePiece joint source/target, 24k
• Suggested decoding: num_beams=4, max_length=512 (or CT2 int8 below), followed by postprocess() from the section above.

Quick Start

import re
from transformers import AutoTokenizer, AutoModelForSeq2SeqLM

model_id = "DanVP/MoxhiMT-30-QT"
tok = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForSeq2SeqLM.from_pretrained(model_id)

_OPENER_SPACE = re.compile("([" + re.escape("“‘«「『【《([{（［｛") + r"]) (?=[^\W\d_])")

def translate(text: str) -> str:
    inputs = tok(text, return_tensors="pt", truncation=True, max_length=512)
    out = model.generate(**inputs, max_length=512, num_beams=4)
    return _OPENER_SPACE.sub(r"\1", tok.decode(out[0], skip_special_tokens=True))

print(translate("你以为呢？我们走吧。"))
# → Ngươi nghĩ sao? Chúng ta đi thôi.
print(translate("“哥，你别管我！”她甩开他的手，“我自己的事自己做主。”"))
# → “Ca, ngươi đừng quản ta!” Nàng hất tay hắn ra, “Chuyện của chính ta tự mình làm chủ.”

Fast CPU Runtime

A CTranslate2 INT8 export is in ct2-int8_float32/ for ~3-5x faster CPU inference.

import ctranslate2
from pathlib import Path
from huggingface_hub import snapshot_download
from transformers import AutoTokenizer

model_id = "DanVP/MoxhiMT-30-QT"
model_path = Path(snapshot_download(model_id, allow_patterns=[
    "config.json", "source.spm", "target.spm", "vocab.json",
    "tokenizer_config.json", "ct2-int8_float32/*",
]))
tokenizer = AutoTokenizer.from_pretrained(model_path)
translator = ctranslate2.Translator(
    str(model_path / "ct2-int8_float32"),
    device="cpu", compute_type="int8_float32",
)

Apply the same postprocess() to CT2 output.

Training Data

Same curated Chinese–Vietnamese web-novel corpus as MoxhiMT-30 v4-base (xianxia, modern, historical, sci-fi, cross-domain + research-grounded idiom / classical-grammar layer). The only change for v1.0: every target was normalized to the QT register by a source-anchored, 78-test rewrite pipeline (per-segment budgeting, proper-noun/idiom guards, audited over two independent 500-sample review rounds). v1.1 adds a small dialogue lane: 6.5k lines with vocatives mined from raw novels (hash-gated against the existing pool, and from different novels than the held-out probe), translated by a teacher model under a strict "pronoun by source character" prompt and filtered by the same leak checker that produced the numbers above.

Notes

• Shares the family's known hard cases: rare proper nouns, highly domain-specific OOD terminology.
• Repetition rate is on par with the non-DPO baseline; a DPO pass may follow.
• Experimental release; review output for high-stakes or publication use.

License

CC-BY-NC-4.0 (research / non-commercial use).