SDXL-Turbo NVFP4 (ModelOpt 0.46.1 PTQ)

NVFP4 post-training quantization of stabilityai/sdxl-turbo via NVIDIA Model-Optimizer (--format fp4 --quantize-mha --calib-size 128 --n-steps 4). UNet fp16 ~5.13 GB -> ~1.96 GB safetensors (~2.6x).

Prompt: beautiful lady portrait, freckles, soft light, photorealistic (4 steps, guidance 0.0, seed 2946901)

Load (designed path)

Plain from_pretrained on the safetensors does NOT work: the NVFP4+MHA export writes restructured weights. Restore the torch checkpoint instead:

import torch, modelopt.torch.opt as mto
from diffusers import AutoModel, StableDiffusionXLPipeline

unet = AutoModel.from_pretrained("stabilityai/sdxl-turbo", subfolder="unet",
                                 torch_dtype=torch.float16)
mto.restore(unet, "sdxl-turbo-nvfp4.pt")  # this repo
pipe = StableDiffusionXLPipeline.from_pretrained(
    "stabilityai/sdxl-turbo", unet=unet, torch_dtype=torch.float16).to("cuda")

Files: unet/ = exported HF checkpoint (SGLang / vLLM-Omni / TRT-LLM compatible layout, quant_algo=NVFP4), sdxl-turbo-nvfp4.pt = ModelOpt torch checkpoint for mto.restore.

NVFP4 inference needs Blackwell GPUs + modelopt kernels (or TRT-LLM/SGLang).