Qwen-Image Nunchaku Lite INT4 R32 with BNB4 Text Encoder

[!IMPORTANT] Experimental: This Nunchaku Lite quantized Qwen-Image checkpoint is experimental. APIs and loading behavior may change.

This repository contains a full Diffusers pipeline for Qwen/Qwen-Image with:

• a Nunchaku Lite quantized diffusion transformer
• a prequantized bitsandbytes 4-bit NF4 text_encoder
• inherited tokenizer, scheduler, and VAE from the Qwen-Image release

Quantization

The transformer uses a compact Nunchaku Lite quantization config stored in transformer/config.json.

Requirements

This checkpoint requires a Diffusers version that includes NunchakuLiteQuantizationConfig and the Nunchaku Lite loader path.

pip install git+https://github.com/huggingface/diffusers
pip install kernels bitsandbytes

Use a CUDA-capable PyTorch environment. The quantized transformer uses kernels from the Hugging Face kernels package, and the text encoder uses bitsandbytes 4-bit modules.

Usage

import torch
from diffusers import QwenImagePipeline

pipe = QwenImagePipeline.from_pretrained(
    "lite-infer/Qwen-Image-nunchaku-lite-int4_r32-bnb4-text-encoder",
    torch_dtype=torch.bfloat16,
).to("cuda")

image = pipe(
    prompt="A cinematic photo of a glass teapot on a wooden table, warm window light, detailed reflections, shallow depth of field",
    negative_prompt=" ",
    height=1024,
    width=1024,
    num_inference_steps=20,
    true_cfg_scale=4.0,
    generator=torch.Generator(device="cuda").manual_seed(42),
).images[0]

image.save("qwen-image-nunchaku-lite-int4_r32-bnb4-text-encoder.png")

Evaluation

Evaluation was run with the following settings:

• Hardware: NVIDIA GeForce RTX 5090 or equivalent Blackwell GPU with CUDA
• Precision: torch_dtype=torch.bfloat16
• Output: 1024x1024
• Inference steps: 20
• Guidance: true_cfg_scale=4.0
• Negative prompt: " "
• Seed: 42
• Runs: 1 warmup run, 3 measured runs

The + torch.compile rows compile only pipe.transformer with mode=None and fullgraph=False.

[!NOTE] On Blackwell GPUs, the INT4 variants measured slower than FP4 in this benchmark. Prefer FP4 variants for latency-sensitive use on Blackwell-class hardware.

The Diffusers integration currently uses the Nunchaku Lite quantized modules and kernels, but does not include the original Nunchaku runtime's kernel/module fusing. The + torch.compile rows compile only the Diffusers transformer graph around those modules, so the speedup can be smaller than the original Nunchaku checkpoint benchmark below.

Original Nunchaku Checkpoint Benchmark

The original Nunchaku transformer was loaded with NunchakuQwenImageTransformer2DModel.from_pretrained(...) and inserted into the matching converted pipeline components so the text encoder, tokenizer, VAE, and scheduler matched the converted checkpoint.

Result Image

Conversion Comparison

Provenance

• Base pipeline: Qwen/Qwen-Image
• Source quantized checkpoint: nunchaku-ai/nunchaku-qwen-image, svdq-int4_r32-qwen-image.safetensors

Limitations

• This is an experimental quantized checkpoint.
• The transformer and text encoder are quantized; the scheduler, tokenizer, and VAE are inherited from the base model.
• Output quality and performance may differ from the original BF16 pipeline and from the original Nunchaku runtime.

License

This repository follows the Apache-2.0 license from the base Qwen-Image release.