MiLMMT-46-4B v1.0 — 4-bit (MLX, text-only)
A 4-bit, group-size-64 MLX quantization of Xiaomi's
MiLMMT-46-4B-v1.0, for on-device
translation on Apple Silicon. 46 languages.
The base checkpoint is a Gemma3ForConditionalGeneration with a SigLIP vision tower. This
conversion is text-only: vision_tower and multi_modal_projector are dropped, leaving the
language model — 8.6 GB (bf16, with vision) down to 2.1 GB.
| Base model | xiaomi-research/MiLMMT-46-4B-v1.0 |
| Quantization | 4-bit, group size 64, affine (4.501 bits/weight) |
| Format | MLX safetensors |
| Size | ~2.1 GB |
| Modality | text-only (vision tower removed) |
| Languages | 46 |
A 5-bit build (~2.5 GB) is also available.
Measured against full precision
The whole ladder was benched against the unquantized bf16 model on FLORES+ devtest, en→X, 45 languages × 100 sentences (4,500 segments per rung), chrF++ against the gold human references. Significance is sacrebleu's paired bootstrap (2,000 resamples) over the pooled 4,500 sentences.
| rung | size | mean chrF++ | Δ vs bf16 (pooled) | p | languages below bf16 |
|---|---|---|---|---|---|
| bf16 (reference) | 7.3 GB | 53.86 | — | — | — |
| 5-bit | 2.70 GB | 53.83 | −0.04 | 0.20 | 23 / 45 |
| 4-bit | 2.22 GB | 53.06 | −0.88 | 0.0005 | 40 / 45 |
5-bit is statistically indistinguishable from full precision — Δ −0.04, 95% CI ±0.43, and a 23/45 language split that is a coin flip. 4-bit costs a real ~0.9 chrF++, significant at p=0.0005 with a 40/45 split, concentrated in the lower-resource tail (Turkish −1.98, Uzbek −1.36, Tamil −1.33).
A blind Codex judge (different model family, scoring 450 items against the same gold references) independently put 4-bit 1.00 points below bf16 on a 0–100 scale — agreeing with chrF++ to within 0.12. A per-sentence scan of all 4,500 outputs per rung found no empty outputs, truncations, or repetition loops at any precision.
Earlier reference-free pairwise judging of 4-bit against 5-bit and 8-bit found no difference. That was a limitation of the method, not a property of the model: comparing two quantized rungs to each other cannot locate a floor, and without a gold reference the judges could not resolve a sub-point gap.
Pick 4-bit only if the 480 MB matters more than ~0.9 chrF++. For quality, use the 5-bit build, which measures level with full precision.
Usage
from mlx_lm import load, generate
from mlx_lm.sample_utils import make_sampler
model, tokenizer = load("translate-studio/MiLMMT-46-4B-v1.0-4bit-MLX")
src, tgt = "Chinese (Simplified)", "English"
text = "我爱机器翻译"
prompt = f"Translate this from {src} to {tgt}:\n{src}: {text}\n{tgt}:"
print(generate(model, tokenizer, prompt=prompt, max_tokens=512,
sampler=make_sampler(temp=0.0), verbose=False))
Prompt format
Use the base model's plain block, with no BOS token and no chat template — the checkpoint's
chat_template.jinja simply concatenates message contents, so there are no turn markers:
Translate this from <source language name> to <target language name>:
<source language name>: <source sentence>
<target language name>:
Language names must be written as the base model card lists them (e.g. Chinese (Simplified),
Chinese (Traditional), English).
Note on stop tokens
config.json carries only eos_token_id: 1, while generation_config.json lists [1, 106]
(106 is <end_of_turn>). Loaders that read the stop id from config.json alone can miss 106
and run on to max_tokens. Honour both.
Supported languages
Arabic, Azerbaijani, Bulgarian, Bengali, Catalan, Czech, Danish, German, Greek, English, Spanish, Persian, Finnish, French, Hebrew, Hindi, Croatian, Hungarian, Indonesian, Italian, Japanese, Kazakh, Khmer, Korean, Lao, Malay, Burmese, Norwegian, Dutch, Polish, Portuguese, Romanian, Russian, Slovak, Slovenian, Swedish, Tamil, Thai, Tagalog, Turkish, Urdu, Uzbek, Vietnamese, Cantonese, Chinese (Simplified), Chinese (Traditional).
Quality is not guaranteed outside these 46. In particular, prompts targeting Telugu, Marathi or Gujarati — which are not in the list — return fluent Hindi/Devanagari rather than an error, at both 4-bit and 5-bit. Filter to the supported set rather than relying on the model to refuse.
License & attribution
Derived from Gemma and distributed under the Gemma Terms of Use.
Gemma is provided under and subject to the Gemma Terms of Use found at ai.google.dev/gemma/terms.
Credit: Xiaomi Inc. (MiLMMT-46) and Google (Gemma 3 base architecture). This repository only provides an MLX-quantized, text-only copy of the released weights, redistributed under the same terms. Use restrictions in the Gemma Terms apply, including the prohibited-use policy.
@misc{han2026referencefreeposttrainingopenlarge,
title={Reference-Free Post-Training of Open Large Language Models for Multilingual Machine Translation},
author={Chris Han and Pengzhi Gao and Pei Fu and Jian Luan},
year={2026},
eprint={2608.10812},
archivePrefix={arXiv},
primaryClass={cs.CL},
url={https://arxiv.org/abs/2608.10812}
}