Qwen3.8 Flash-Next Uncensored NVFP4 + MTP
This is a load-tested sibling of mazinb/Qwen3.8-Flash-Next-Uncensored-NVFP4 with the model's one-layer MTP draft experts converted to NVFP4. It is a roughly 125B-total / 6B-active multimodal mixture-of-experts checkpoint intended for high-throughput speculative decoding on NVIDIA Blackwell.
What is quantized
- All 48 target-model routed MoE expert banks: NVFP4 (inherited from the source checkpoint)
- MTP routed experts (512) and the MTP shared expert: NVFP4
- Target-model shared experts: BF16
- PLE n-gram tables: BF16, qualified with pinned CPU residency and host-to-device lookup
- Attention, linear-attention/GDN, vision, embeddings, gates, and other excluded modules retain the source checkpoint's declared precision
The conversion is deterministic ModelOpt-compatible RTN with group size 16 and no calibration dataset. MTP_NVFP4_MANIFEST.json records source identity, conversion details, preservation closure, and tool version. SHA256SUMS covers every distributed model file; its SHA-256 is 17e1223cb7670c85e52ad56f243bf2702a5325f4ed2e94693a23b4ffedba0292.
Qualified runtime
The exact checkpoint passed end-to-end vLLM qualification on one NVIDIA RTX PRO 6000 Blackwell 96 GB GPU with:
- 88.9 GiB hard VRAM budget, preserving at least 6 GiB physical VRAM
- 131,072-token serving context
- MTP speculative depth 3
- BF16 KV cache with PLE-only CPU offload
- 161.12 output tokens/s in the gated single-stream decode-after-prefill test
- 974 accepted draft tokens out of 1,689 observed during qualification
- 120,031-token long-context, multimodal, tool-call, reasoning-channel, readiness, behavior, and resource checks
This architecture requires a recent compatible vLLM build with Qwen4-Exp/Flash-Next, Blackwell NVFP4 MoE, MTP, and PLE CPU-offload support. Stock older vLLM releases may not load it. Use trust_remote_code only if your runtime policy permits it, and verify SHA256SUMS before serving.
Lineage
- Uncensored NVFP4 parent:
mazinb/Qwen3.8-Flash-Next-Uncensored-NVFP4at revisionf2c21eb3d2ff5f24c208ea7e3afba65e2e70f83f - The parent model card documents its behavioral merge/source lineage.
- This repository changes MTP expert storage and the one source shard that formerly contained superseded BF16 MTP expert tensors; all other checkpoint tensors are byte-preserved.
Safety and use
This model is intentionally abliterated/uncensored and may comply with unsafe, illegal, inaccurate, or harmful requests. It has no warranty. Users are responsible for evaluation, access controls, applicable law, and downstream use.
Donations
If this release is useful, Bitcoin donations are welcome:
bc1q5ayht3fxhj0v95fk0z8l2f6900g3awdsw5842p