gguf quantized version of krea

• run it straight with gguf-connector
• opt a gguf file in the current directory to interact with by:

ggc k

GGUF file(s) available. Select which one to use:

1. flux-krea-lite-q2_k.gguf
2. flux-krea-lite-q4_0.gguf
3. flux-krea-lite-q8_0.gguf

Enter your choice (1 to 3): _

note: try experimental lite model with 8-step operation; save up to 70% loading time

• run it with diffusers (see example inference below)

import torch
from transformers import T5EncoderModel
from diffusers import FluxPipeline, GGUFQuantizationConfig, FluxTransformer2DModel

model_path = "https://huggingface.co/calcuis/krea-gguf/blob/main/flux1-krea-dev-q2_k.gguf"
transformer = FluxTransformer2DModel.from_single_file(
    model_path,
    quantization_config=GGUFQuantizationConfig(compute_dtype=torch.bfloat16),
    torch_dtype=torch.bfloat16,
    config="callgg/krea-decoder",
    subfolder="transformer"
)

text_encoder = T5EncoderModel.from_pretrained(
    "chatpig/t5-v1_1-xxl-encoder-fp32-gguf",
    gguf_file="t5xxl-encoder-fp32-q2_k.gguf",
    torch_dtype=torch.bfloat16
    )

pipe = FluxPipeline.from_pretrained(
    "callgg/krea-decoder",
    transformer=transformer,
    text_encoder_2=text_encoder,
    torch_dtype=torch.bfloat16
    )
pipe.enable_model_cpu_offload() # could change it to cuda if you have good gpu

prompt = "a pig holding a sign that says hello world"
image = pipe(
    prompt,
    height=1024,
    width=1024,
    guidance_scale=2.5,
).images[0]
image.save("output.png")

run it with gguf-node via comfyui

• drag krea to > ./ComfyUI/models/diffusion_models
• drag clip-l-v2 [248MB], t5xxl [2.75GB] to > ./ComfyUI/models/text_encoders
• drag pig [168MB] to > ./ComfyUI/models/vae

reference

• base model from black-forest-labs
• for model merge details, see sayakpaul
• diffusers from huggingface
• comfyui from comfyanonymous
• gguf-node (pypi|repo|pack)
• gguf-connector (pypi)