FLUX.1 Kontext Dev Nunchaku Lite NVFP4 r32
Diffusers-loadable conversion of:
- Base model:
black-forest-labs/FLUX.1-Kontext-dev - Source repo:
nunchaku-ai/nunchaku-flux.1-kontext-dev - Source checkpoint:
svdq-fp4_r32-flux.1-kontext-dev.safetensors
The CLIP text encoder is copied from the base model, and the T5 text_encoder_2 is saved with BitsAndBytes 4-bit NF4.
Benchmark
| Checkpoint | Latency | Max VRAM |
|---|---|---|
| Converted Diffusers Nunchaku Lite NVFP4 r32 + BNB4 T5 encoder | 9.96 s (stdev 0.07 s) | 14.19 GiB |
| Original Nunchaku NVFP4 r32 safetensors (native Nunchaku engine, stock bf16 T5) | 5.89 s (stdev 0.01 s) | 18.90 GiB |
The native engine's latency advantage comes from hand-written CUDA kernels and a purpose-built pipeline path, versus the Diffusers Nunchaku Lite quantizer's more general module-replacement path — text encoder choice barely affects per-run latency since T5 encoding happens once per call, not per step. Its higher VRAM here is entirely the bf16 T5 (~9.5 GB) vs. this repo's NF4 T5 (~5 GB); with a BNB4 T5 substituted into the native pipeline instead, its peak VRAM measured 14.13 GiB, matching the converted pipeline.
Output Comparison
The comparison uses the same input image, prompt, seed, scheduler, and 28-step settings for this converted checkpoint and the original Nunchaku safetensors checkpoint run through the native Nunchaku engine with its stock bf16 T5 (same GPU and session as the benchmark above).
Run
Requires a Blackwell or newer NVIDIA GPU (NVFP4 kernels) and the Hugging Face kernels package.
import torch
from PIL import Image
from diffusers import FluxKontextPipeline
model_path = "lite-infer/flux.1-kontext-dev-nunchaku-lite-nvfp4_r32-bnb4-text-encoder"
image_path = "input.png"
output_path = "output.png"
pipe = FluxKontextPipeline.from_pretrained(model_path, torch_dtype=torch.bfloat16)
pipe.to("cuda")
image = Image.open(image_path).convert("RGB")
prompt = "Make Pikachu hold a sign that says 'Nunchaku is awesome', yarn art style, detailed, vibrant colors"
result = pipe(
image=image,
prompt=prompt,
generator=torch.Generator(device="cuda").manual_seed(1),
true_cfg_scale=1.0,
guidance_scale=2.5,
num_inference_steps=28,
max_area=512 * 512,
)
result.images[0].save(output_path)