Z1T-0 (MLX)
MLX conversion of Extropic-AI/Z1T-0 — the first open-weight Z1T, an attention-free, DyT-normed, fixed-sparsity decoder LM designed for Extropic's Z1 probabilistic hardware (research post, code).
The original is a JAX/Equinox checkpoint (.eqx) with a bespoke architecture (no softmax
attention: causal depthwise conv + causal cumulative pool; tanh-linear fixed fan-in-4 sparse
projections; Dynamic-Tanh as the only norm). This repo ports it to MLX and ships it as
safetensors + a GPT-2 tokenizer.
Faithfulness: greedy decoding is token-for-token identical to the JAX reference; logits
match to max|Δ|≈1.4e-2 (float32 accumulation-order noise), with 100% per-position argmax
agreement.
Files
model.safetensors— weights (float32, ~1.24B params; dominated by the 50257×12288 embed/unembed).config.json—model_type: z1t+ architecture fields.tokenizer.json/vocab.json/merges.txt/tokenizer_config.json— standard GPT-2 BPE.mlx_z1t.py,run.py— self-contained MLX inference (no JAX, no mlx-vlm needed).
Usage — standalone (no dependencies beyond MLX)
pip install mlx numpy tiktoken safetensors
python run.py "The meaning of life is" 60
Usage — via mlx-vlm
Requires the z1t model in mlx-vlm (mlx_vlm/models/z1t/):
from mlx_vlm import load, generate
model, processor = load("AlazarM/Z1T-0-mlx")
print(generate(model, processor, "The meaning of life is", max_tokens=60, temperature=0.0).text)
Notes
- This is a portability port (runs on Apple Silicon), not a throughput build. Prefill is ~150 µs/token on an M5 Max; single-token decode is unoptimized.
- The checkpoint ships a numerically-zero positional table (AFTConv already encodes position).
- The model is a small 4-layer research artifact; generations are short and repetitive.
Original weights © Extropic, Apache-2.0. This conversion redistributes them unmodified in MLX form.