gguf quantized version of qwen-image
- run it straight with
gguf-connector
ggc q5
GGUF file(s) available. Select which one to use:
- qwen-image-iq2_s.gguf
- qwen-image-iq4_nl.gguf
- qwen-image-q4_0.gguf
- qwen-image-q8_0.gguf
Enter your choice (1 to 4): _
run it with gguf-node via comfyui
- drag qwen-image to >
./ComfyUI/models/diffusion_models - drag qwen2.5-vl-7b [4.43GB] to >
./ComfyUI/models/text_encoders - drag pig [254MB] to >
./ComfyUI/models/vae
tip: the text encoder used for this model is qwen2.5-vl-7b; get more encoder either here (pig quant) or here (llama.cpp quant); the size is different from the one (qwen2.5-vl-3b) used in omnigen2
run it with diffusers
import torch
from diffusers import DiffusionPipeline, GGUFQuantizationConfig, QwenImageTransformer2DModel
model_path = "https://huggingface.co/calcuis/qwen-image-gguf/blob/main/qwen-image-q2_k.gguf"
transformer = QwenImageTransformer2DModel.from_single_file(
model_path,
quantization_config=GGUFQuantizationConfig(compute_dtype=torch.bfloat16),
torch_dtype=torch.bfloat16,
config="callgg/qi-decoder",
subfolder="transformer"
)
pipe = DiffusionPipeline.from_pretrained(
"callgg/qi-decoder",
transformer=transformer,
torch_dtype=torch.bfloat16,
)
pipe.enable_model_cpu_offload()
prompt = "a pig holding a sign that says hello world"
positive_magic = {"en": "Ultra HD, 4K, cinematic composition."}
negative_prompt = " "
image = pipe(
prompt=prompt + positive_magic["en"],
negative_prompt=negative_prompt,
height=1024,
width=1024,
num_inference_steps=24,
true_cfg_scale=2.5,
generator=torch.Generator()
).images[0]
image.save("output.png")
note: diffusers not yet supported t and i quants; opt gguf-node via comfyui or run it straight with gguf-connector
reference
- base model from qwen
- distilled model from modelscope
- lite model is a lora merge from lightx2v
- comfyui from comfyanonymous
- diffusers from huggingface
- gguf-node (pypi|repo|pack)
- gguf-connector (pypi)