Nanbeige4.2-3B — MLX bf16
Full-precision bfloat16 weights (7.8 GB).
Converted from Nanbeige/Nanbeige4.2-3B using mlx-lm. Logits of the MLX port match the reference implementation to a max abs difference of 9.5e-05 (fp32, CPU).
Nanbeige4.2-3B is a Looped Transformer: its 22 decoder layers run twice per
forward pass (num_loops=2) with shared weights, giving 4B total parameters
with the capacity of a deeper model.
Requirements
The nanbeige model type is pending upstream support in mlx-lm
(ml-explore/mlx-lm#1597).
Until that merges, install from the PR branch:
pip install git+https://github.com/MercuriusDream/mlx-lm.git@add-nanbeige-model
Use
mlx_lm.generate --model MercuriusDream/Nanbeige4.2-3B-mlx-bf16 --prompt "hello"
from mlx_lm import load, generate
model, tokenizer = load("MercuriusDream/Nanbeige4.2-3B-mlx-bf16")
prompt = tokenizer.apply_chat_template(
[{"role": "user", "content": "hello"}], add_generation_prompt=True
)
print(generate(model, tokenizer, prompt=prompt))