MiMo-V2.6-Flash-RL: exact NVFP4 weight conversion

This converts the released MXFP4 routed experts to NVFP4 without changing any expert weight value. There is no calibration, clipping, optimization, or nearest-code rounding in this conversion. It is a format transcode of the already quantized upstream checkpoint, not a new 4-bit quantization of BF16 weights. Hugging Face's quantized base-model relation describes the stored format; it does not mean an additional lossy quantization was performed here.

Source: XiaomiMiMo/MiMo-V2.6-Flash-RL, revision 3b38d063180c3e4aed9691fdc735f3d10b266ee4. The original model and tokenizer remain subject to the upstream MIT license.

What changed

• Every E2M1 weight code remains unchanged.
• Each E8M0 scale for 32 weights becomes two exact E4M3 scales for 16 weights, combined with an exact power-of-two global scale. All scale groups were checked for representability; none required rounding.
• Dense FP8 tensors are reconstructed in FP32 to preserve their decoded values. Fused attention weights are reordered from the checkpoint's TP=4 interleaving to global Q/K/V order, including its per-rank scale padding.
• Other indexed BF16/F32 tensors retain their bytes. Multimodal encoders, audio tokenizer, tokenizer files, and the upstream DFlash sidecars are retained.
• The upstream DFlash configuration's trailing comma is repaired. Its parsed configuration is unchanged.
• ModelOpt mixed-precision metadata selects W4A16_NVFP4 for the experts. No FP4 activation quantization is requested by this checkpoint configuration.

The root tensor payload is 193,964,338,432 bytes, excluding auxiliary files. The larger dense tensors are deliberate: preservation takes priority over reducing every tensor to a low-precision format.

Verification

verification.json records an independent, complete audit of all 65 root shards:

Source shard SHA-256 values were also compared with the pinned Hugging Face LFS manifest. auxiliary-verification.json covers the retained auxiliary files. Reproduction and independent verification scripts are in tools/.

Exact stored weights do not imply bit-identical inference. Runtime dtype, activation quantization, attention kernels, accumulation, and sampling can change outputs. In particular, loading FP32 reconstructions with BF16 compute rounds them to the runtime dtype. Native Blackwell FP4 activation kernels must be evaluated separately from this weight conversion. No 60-token/s CPU-offload result is claimed by the weight audit.

Held-out runtime comparison

The original MXFP4 GGUF and converted NVFP4 GGUF were compared using the same pinned llama.cpp revision, cuBLAS F32 storage/accumulation, and BF16 KV. The runtime's cuBLAS handle enables TF32 math for eligible large multiplications; GGML_CUDA_CUBLAS_COMPUTE_TYPE=f32 alone does not disable that setting. Both original and converted models used the same math mode. The historical strict report label means FP4/Q8 activation kernels were bypassed, not that every multiplication used full-mantissa FP32. See NVIDIA's TF32 controls. All original dense values remain FP32. The evaluation covers eight 4,096-token chunks per domain, scoring the last 2,047 positions in each chunk: 65,504 scored token positions across code, math, English and Chinese.

Perplexity matches within the precision of the original PPL logs. Mean approximate KL is zero to six printed decimals in every domain. The reference log probabilities are encoded by llama.cpp as uint16 values with a max_logit - 16 floor; ties from this encoding can affect the top-token diagnostic. PPL ratios use the original unclipped PPL logs, not the clipped log-probability file. These measurements do not establish a general task accuracy percentage or certify other inference kernels.

See reports/heldout-nvfp4-strict.json, the associated logs, and reports/gguf-weight-verification.json. The latter independently verifies all 302,795,194,368 expert values and all 6,971,406,720 dense values in the two GGUF exports. This canonical release contains safetensors and an independently verified GGUF package. Workstation runtime and performance tests are reported separately.

Tested loading path

The checkpoint loads with vLLM 0.29.0, tensor parallelism 2, on two RTX PRO 6000 Server Edition 96 GB GPUs. A basic arithmetic completion and executable coding checks have run. This is a runtime smoke test, not a broad quality benchmark. CPU offloading and speculative decoding require separate validation.

vllm serve ProCreations/MiMo-V2.6-Flash-RL-NVFP4 \
  --served-model-name mimo-v26-nvfp4 \
  --tensor-parallel-size 2 --dtype bfloat16 \
  --max-model-len 12288 --max-num-seqs 2 \
  --max-num-batched-tokens 512 --gpu-memory-utilization 0.94 \
  --enforce-eager --generation-config vllm \
  --reasoning-parser mimo --tool-call-parser mimo --enable-auto-tool-choice \
  --host 127.0.0.1 --port 30080

Use the upstream recommended sampling settings explicitly: temperature 1.0, top_p 0.95. The original generation_config.json is retained for provenance and includes do_sample: false; API requests should explicitly enable the intended sampling settings. Long reasoning can exhaust response limits, so count truncated responses as incomplete when evaluating real work.

The canonical safetensors release retains the original multimodal and DFlash weights. Inclusion does not certify every runtime's vision/audio generation or DFlash implementation. See the accompanying reports and recipes for the exact tested configurations and outstanding limitations.

Single-workstation FP8 KV runtime

The GGUF runtime recipe provides one CPU-offloaded profile for a 96 GB RTX PRO 6000 with 128 GB-class host RAM: 524,288 total context tokens, 131,072 maximum output tokens, scaled E4M3 FP8 target KV, DFlash, and active-prefix reuse. See measured speeds, completed work and limitations. Near-512K retrieval failed and sustained 40–60 tok/s across workloads was not achieved. The weight conversion remains exact; FP8 KV and runtime arithmetic have separate numerical checks.