Nanbeige4.2-3B — MLX (bfloat16)
An MLX bfloat16 conversion of
Nanbeige/Nanbeige4.2-3B,
ready to run on Apple Silicon with mlx-lm.
- Format: MLX safetensors,
bfloat16(no quantization — full-precision weights) - Size: ~8.3 GB · ~4.2B stored parameters
- Converted from:
Nanbeige/Nanbeige4.2-3Bwithmlx_lm.convert(..., quantize=False, dtype="bfloat16")
[!IMPORTANT] This model needs a custom architecture file to load. Nanbeige uses a looped transformer architecture (
model_type: "nanbeige") thatmlx-lmdoes not support out of the box. Without the step below you will get:ValueError: Model type nanbeige not supported.This repo ships the requirednanbeige.py— see Setup.
Setup
Install mlx-lm, then drop the bundled nanbeige.py into mlx-lm's model registry:
pip install -U mlx-lm
# Download nanbeige.py from this repo, then copy it into mlx_lm/models/:
python - <<'PY'
import mlx_lm, os, shutil
dst = os.path.join(os.path.dirname(mlx_lm.__file__), "models", "nanbeige.py")
shutil.copy("nanbeige.py", dst)
print("installed ->", dst)
PY
Note: re-installing or upgrading
mlx-lmremoves this file — just copy it again.
Usage
Python
from mlx_lm import load, generate
model, tokenizer = load("cudo528/Nanbeige4.2-3B-mlx-bf16")
prompt = tokenizer.apply_chat_template(
[{"role": "user", "content": "What is the capital of France?"}],
add_generation_prompt=True,
)
text = generate(model, tokenizer, prompt=prompt, max_tokens=256, verbose=True)
Command line
mlx_lm.generate \
--model cudo528/Nanbeige4.2-3B-mlx-bf16 \
--prompt "Explain rotary position embeddings in one paragraph."
Model details
| Base model | Nanbeige/Nanbeige4.2-3B |
| Architecture | nanbeige — Llama-style decoder run as a looped transformer |
| Decoder layers | 22 (weights) · executed 2× (num_loops = 2) → 44 effective passes |
| Hidden size | 3072 |
| Attention heads | 48 (head_dim = 128) |
| KV heads | 8 (grouped-query attention) |
| Intermediate size | 10752 (SwiGLU / SiLU) |
| Normalization | RMSNorm (eps = 1e-5) |
| Position encoding | RoPE (theta = 7e7) |
| Vocab size | 166,144 |
| Max context | 262,144 |
| Tied embeddings | No (separate lm_head) |
| Weight dtype | bfloat16 |
About the looped architecture
Nanbeige reuses a single 22-layer stack for num_loops passes instead of stacking
distinct layers. The effective forward is:
h = embed(x)
for loop in range(num_loops): # num_loops = 2
for layer in layers: h = layer(h) # KV cache slot = layer_idx + loop * 22
h = norm(h) # final norm after each loop pass
logits = lm_head(h)
Because layers are shared across loops, the KV cache holds
num_loops × num_layers = 44 entries (each loop pass keeps its own keys/values).
This behavior is implemented in the bundled nanbeige.py.
Conversion
from mlx_lm import convert
convert(
"Nanbeige/Nanbeige4.2-3B",
"Nanbeige4.2-3B-mlx-bf16",
quantize=False,
dtype="bfloat16",
)
The converted model loads in mlx-lm and produces coherent generations (e.g.
correctly answers "The capital of France is Paris."). Weights are cast to
bfloat16 with no quantization, so quality matches the original within bf16 precision.
Credits & license
- Original model & weights: Nanbeige/Nanbeige4.2-3B by the Nanbeige team.
- This is an unofficial MLX conversion; all credit for the model goes to the original authors.
- Released under Apache-2.0, following the base model's license. Please also review the base model's usage terms.