NVIDIA-Nemotron-3.5-Lightning-30B-A3B-DSpark (bf16, MLX)
The DSpark speculative-decoding drafter for
nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B,
dequantized to bf16 for use on Apple Silicon with
mlx-dspark.
This is a format conversion, not a retrain. NVIDIA ships the drafter as
nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4-DSpark
in NVIDIA Model Optimizer NVFP4 (W4A16_NVFP4), which stock MLX / mlx-lm cannot read. The
FP4 (E2M1) weights + FP8 (E4M3) block scales were hand-decoded back to bf16 (attention /
embed_tokens / markov_w1 were already bf16 in the source). Weights are otherwise unchanged;
mlx-dspark re-quantizes the drafter to 4-bit at load, which does not affect acceptance.
Use with mlx-dspark
Pair it with the 4-bit MLX target
mlx-community/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-4bit:
pip install mlx-dspark
mlx-dspark generate \
--model mlx-community/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-4bit \
--drafter mlx-community/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-DSpark-bf16 \
--max-draft 4 --prompt "Write a Python function for the n-th Fibonacci number."
The output is lossless — the target verifies every token, so speculation only changes speed, never the text (it matches greedy target decoding up to floating-point ties).
Architecture
A plain Qwen3 GQA drafter backbone (6 layers, hidden 2688, 32 heads / 2 KV, head_dim 128,
vocab 131072) with the DSpark rank-512 Markov fixup head and DFlash-lineage traits: causal
sliding-window (1024) block attention with a per-head attention-sink bias, block_size 8,
sample_from_anchor=false (predictions start at block slot 1). It ships its own embed_tokens
and reuses the target's lm_head (has_lm_head=false). 967M params total (615M
non-embedding). Fuses 6 target hidden-state taps (layers 1, 5, 19, 29, 41, 51).
The target is a hybrid Mamba-2 + latent-MoE + attention model (model_type: nemotron_h).
mlx-dspark adds the nemotron_h hidden-state tap and an exact Mamba-2 recurrent-state rollback so
the lossless verify loop works on it.
Measured (M4 Pro, mlx 0.32, greedy, warm)
Baseline (4-bit target) ≈ 91 tok/s. DSpark at --max-draft 4 (lookup off):
| content | speedup | accept length |
|---|---|---|
| code | 1.27× (→ ~117 tok/s) | 4.55 |
| math | 1.24× | 4.41 |
| chat | 1.06× | 3.74 |
Throughput peaks at cap 4 and regresses past it (the MoE verify-width cost climbs faster than acceptance) — the speedup here is bounded by the MoE, not the drafter, whose acceptance is strong.
License & attribution
Original drafter © NVIDIA, released under the
OpenMDW-1.1 model license (see LICENSE). This repository
redistributes a dequantized (NVFP4 → bf16) copy under the same terms. Produced with
NVIDIA Model Optimizer (source) and
mlx-dspark (conversion).