Ornith-1.0-35B-MTPLX-Vision

A unified local model that both codes at the Ornith-1.0-35B level and sees images/videos, packaged for Apple Silicon via MTPLX native speculative decoding with multi-token-prediction (MTP).

This repository restores the multimodal vision tower that is typically stripped in text-only MLX/MTPLX conversions.

Architecture

• Body: 4-bit affine, group size 64 (from wang-yang/Ornith-1.0-35B-MTPLX)
• MTP sidecar (mtp.safetensors): 4-bit quantized MTP heads.
• Vision tower (vision_tower.safetensors, 333 tensors, bf16 unquantized): Extracted verbatim from the official checkpoint's own model.visual.* weights (shard 16 of deepreinforce-ai/Ornith-1.0-35B) and renamed to the vision_tower.* prefix for MTPLX native VLM compatibility (~852 MB).

Usage (MTPLX)

mtplx start --model <path-to-this-model-dir> --port 8092 \
  --chat-template-path <path-to-this-model-dir>/chat_template.jinja

Serves an OpenAI-compatible endpoint supporting both text and image input payloads via POST /v1/chat/completions.

The MTPLX CLI defaults to mtp_history_policy=committed, which this model requires for healthy deep-position acceptance. For best throughput on short prompts also pass --draft-temperature 0 (greedy draft).

Performance (measured)

Apple M5 Pro 64 GB, MTPLX 1.0.4, temperature 0.6, mtp_history_policy=committed, --draft-temperature 0, thinking OFF, warm (4 warmup prompts discarded), 8 measured prompts from the calibration_coding suite, max_tokens=192. Median over 3 independent runs.

Two throughput definitions are reported because they answer different questions:

• tok/s (wall-clock e2e) = generated_tokens / total_elapsed, includes prompt prefill. The real end-to-end speed a user experiences and the honest denominator for the speedup ratio.
• tok/s (decode-only) = generated_tokens / decode_elapsed, excludes prefill. Comparable to the older 2026-06-27 M3 Max figures that used decode-only throughput.

Acceptance is the stable MTP health signal. Across 3 independent runs the per-position acceptance was rock-stable: pos1 0.85-0.87, pos2 0.69-0.71, pos3 0.52. Tok/s varies run-to-run (AR e2e bounces 35-60 tok/s depending on thermal/contention state), so the speedup ratio inherits that noise; the acceptance numbers are the reliable indicator that the MTP graft is healthy.

The earlier README reported 1.06x; that number used a single cold trial with the non-default mtp_history_policy=cycle, which collapses pos3 acceptance to ~0.27. With the committed history policy that the CLI already serves by default, deep acceptance recovers and MTP depth 2/3 win clearly over AR.

Why this vision graft works (and the non-vision source does not)

The non-vision source (wang-yang/Ornith-1.0-35B-MTPLX) collapses to pos1 ~= 0.23 acceptance on M5 Pro under every history policy tested (committed, cycle, full, last_window, with and without greedy draft). This is not a policy or loader bug. A direct weight diff shows the cause:

• The MTP draft head (mtp.safetensors) is bit-for-bit identical between wang-yang and this vision variant (46/46 leaf tensors, 0 differences).
• The MTP head's own gate quantization is 4-bit in both.
• The target body, however, differs in 264 of 1803 leaf tensors (concentrated in linear_attn, MoE gates, shared_expert, switch_mlp across layers 11-33). The two models are not the same weights with a vision tower bolted on; the vision variant's body was re-derived such that the shipped draft head matches it, while wang-yang's body does not.

So the published wang-yang draft head diverges from wang-yang's own target body, and MTP acceptance collapses. This vision variant pairs a body whose weights match the shipped draft head. The vision tower itself is lazy-loaded (only when a request carries an image), so text-only throughput is unaffected by it.

A higher-acceptance alternative

For the best MTP acceptance on this family, the Shiftedx/ornith-1.0-35b-abliterated-mxfp4-vision-mtplx variant (mxfp4 body, 5-bit MTP head) measures, on the same M5 Pro 64GB under the same committed+greedy config:

Its pos3 acceptance (0.76) is the highest of the Ornith MTP variants tested. Note: loading that variant on stock MTPLX <= 2.1.0 requires a one-line fix to _infer_prequantized_group_size (it miscomputes the group size for 5-bit MTP heads); see MTPLX issue #182 / PR #183.

Numbers are indicative, not a benchmark suite.

Sources And Attribution

This derivative is released under MIT, preserving the base model's license. The grafted MTP tensors originate from the Apache-2.0 licensed Qwen3.5-35B-A3B; that license and its NOTICE apply to those tensors.