Qwen3.8-2.4T-A95B-NVFP4
nvfp4 quantization of Qwen/Qwen3.8-2.4T-A95B, produced with compressed-tensors by streaming the checkpoint tensor-by-tensor (the model is never fully instantiated).
NVFP4 4-bit float weights in groups of 16 with an FP8 group scale and a per-tensor global scale. Roughly half the size of FP8 at moderate error.
All quantizations of this model
| Variant | Format | Size | vs BF16 | Mean rel. error | Linears quantized | Left BF16 |
|---|---|---|---|---|---|---|
| Qwen3.8-2.4T-A95B-FP8 | float-quantized | 2453.05 GB | 50% | 0.0264 | 143569 | 0 |
| Qwen3.8-2.4T-A95B-NVFP4 ← this one | nvfp4-pack-quantized | 1382.45 GB | 28% | 0.0952 | 143569 | 0 |
| Qwen3.8-2.4T-A95B-int4 | pack-quantized | 1268.00 GB | 26% | 0.1118 | 143569 | 0 |
Mean relative error is ||dequant(W) - W|| / ||W||, averaged over a sample of quantized Linear layers, measured against the original BF16 weights. Lower is better.
This variant
| Format | nvfp4-pack-quantized |
| Weight bits | 4 |
| Group size | 16 |
| Strategy | tensor_group |
| Linears quantized | 143569 |
| Left in BF16 | 0 |
| Shards | 328 |
| On disk | 1382.45 GB |
| Mean relative error | 0.0952 |
| Shape/dtype conformance failures | 0 |
Use with vLLM
vllm serve dudeman2512/Qwen3.8-2.4T-A95B-NVFP4
How this was made
Every produced tensor is checked for shape/dtype conformance against what the server expects, then reconstruction error is measured against the source BF16 weights, before anything is published. The numbers in the table above are those measurements — not estimates.