Ornith-1.0-35B-4bit
4-bit (group size 64, 4.649 bits/weight) MLX quantization of
deepreinforce-ai/Ornith-1.0-35B,
produced with mlx-vlm 0.6.3. Full multimodal: the vision encoder is preserved and
quantized alongside the language model. For Apple Silicon. Runs in mlx-vlm or any MLX app.
Conversion note (MoE expert fusion)
Ornith stores its 256 MoE experts unfused (per-expert), but mlx-vlm's qwen3_5_moe loader expects
them fused/batched. A sanitize monkeypatch was required to stack the experts before conversion; without it the
conversion failed. This is a standard mlx-vlm 4-bit quant.
Usage
uvx --from mlx-vlm mlx_vlm.generate \
--model mlx-community/Ornith-1.0-35B-4bit --image image.png \
--prompt "Describe this image." --max-tokens 512
from mlx_vlm import load, generate
model, processor = load("mlx-community/Ornith-1.0-35B-4bit")
Conversion check
Smoke-tested after conversion (text-only prompt, mlx_vlm.generate): coherent — solved
17 * 24 = 408 with correct step-by-step reasoning, no repetition loop.
103.7 tok/s generation, 89.4 tok/s prompt, peak 20.9 GB on a Macbook Pro M5 Max 128GB 40 GPU.
Refer to the original model card for architecture, benchmarks, license, and intended use.