Qwen-Image-2.1 W4A4 (nvfp4)
Built with Qwen.
Non-commercial use only. An educational artifact under the Qwen Research License, which the base model carries. Commercial use needs a separate licence from the rights holder; publishing this here does not grant one.
4-bit weights, 4-bit activations, for the Qwen-Image-2.1 transformer. Calibrated with SVDQuant, error-compensated with GPTQ, packed for Nunchaku's fused kernel. 4.54 GB against about 10 GB for the same layers in bf16.
The low-rank branch is fused into the 4-bit GEMM, which is where the speed comes from — an unfused branch gives back 24–34% of it.
Hardware
Blackwell only (sm_120 and up). This will not load on Ada — use the int4 build.
Measured
| card | cap | seconds | vs bf16 | resident GB | LPIPS vs bf16 |
|---|---|---|---|---|---|
| RTX PRO 6000 Max-Q | sm_120 | 7.747 | 1.71x | 21.53 | 0.1425 |
| RTX 5090 | sm_120 | 7.653 | — | 21.53 | — |
40 steps, 1024×1024, seed 42, six prompts. LPIPS is AlexNet against the bf16 image of the same prompt on the same card.
GPTQ is worth 23% here: 0.1858 with round-to-nearest against 0.1425 with GPTQ, at the same speed, the same file size and the same resident memory — it only changes which grid point each weight lands on.
Read LPIPS as distance from the bf16 trajectory, not as damage. On a dense-text prompt every build renders the text correctly while scoring 0.24–0.35, and on one prompt the bf16 reference garbles a sign that this checkpoint renders correctly — and is scored worse for it.
Only the transformer is quantized. The Qwen3-VL text encoder stays bf16 and is about 16.3 GB of a 21.5 GB resident total, so it, not this file, decides how many reference images fit on a card.
Text accuracy against bf16
Every LPIPS number above is measured against bf16, so it cannot answer "is this as good as bf16" — it defines bf16 as zero. This is the reference-free version: 8 seeds on each of two text prompts, scored by reading the rendered text against what the prompt asked for.
| build | enamel notice board | neon shop sign | total |
|---|---|---|---|
| bf16 | 8/8 | 6/8 | 14/16 |
| this checkpoint | 8/8 | 7/8 | 15/16 |
Indistinguishable. One render's difference out of sixteen, which a different seed set would flip. The errors trade places rather than accumulating on one side: on one seed bf16 renders the sign as "QWEN 2.1 IMAGE 1" and this checkpoint gets it right; on another this checkpoint appends a spurious "1$" where bf16 is clean.
That is the claim worth making — 4-bit weights and activations with no measurable text degradation — and not the stronger one. A single prompt where the quantized build beats bf16 is trajectory variance, not evidence of superiority.
An automated OCR pass was tried first and discarded: tesseract returns noise on neon at night, and a character-error-rate above 1.0 was the giveaway that it was scoring incidental scene text rather than the sign. The table above comes from reading all 48 renders.
samples/ holds one render per prompt at seed 42. seeds/ holds the full sweep.
Requirements
Packed for a specific kernel. The layout is an MMA fragment swizzle, so a different Nunchaku build may read it incorrectly — and would produce a wrong image rather than an error. The loader checks and warns.
torch==2.12.1
nunchaku==1.3.0.dev20260306+cu13.0torch2.12
Loading
import torch
from diffusers import QwenImage21Pipeline
from quantize import nunchaku_io
transformer = nunchaku_io.load_checkpoint(precision="nvfp4", device="cuda")
pipe = QwenImage21Pipeline.from_pretrained(
"Qwen/Qwen-Image-2.1", torch_dtype=torch.bfloat16, transformer=transformer)
nunchaku_io and the build pipeline: https://github.com/ModelsLab/qwen-image-2-1-server
How it was built
Smoothing factors from real activation statistics (SmoothQuant, α=0.5), a rank-128 SVD branch kept in bf16, and the residual rounded onto Nunchaku's grid by GPTQ against a calibration Hessian. One Hessian pass, 128 distinct inputs across 224 layers, 24 GB resident, ~5 minutes.
w4a4_build.json records the whole build, including per-layer GPTQ statistics.
License
Derivative of Qwen/Qwen-Image-2.1 under the
Qwen Research License Agreement — non-commercial only. See NOTICE.