Qwen-Image-2.1-viggle-turbo — v0.2 (preview)

Built with Qwen. A few-step distilled student of Qwen/Qwen-Image-2.1, trained by Viggle with Distribution Matching Distillation. It does both text-to-image and instruction-driven editing with 1–3 reference images in 5 transformer passes instead of 40, with no classifier-free guidance.

v0.2 (2026-09-23) — use this one:

• Qwen-Image-2.1-viggle-turbo-v0.2-5step-lora-r256.safetensors — LoRA adapter (rank 256, alpha 256, bf16, 1.3 GB) loaded on top of the base transformer at runtime. Sample it with 5 steps on the shipped sigma nodes sigmas=[1.0, 0.875, 0.75, 0.5, 0.25] (see Usage). This is what the demo Space runs.

What changed against v0.1, measured on our held-out set of 96 user requests (text-to-image and editing) against the 40-step base model with its official prompt enhancement:

¹ mean intra-prompt DINOv2 patch distance over 8 seeds per prompt, 32 prompts, as a ratio to the 40-step base model (1.00 = as diverse as the base). v0.1 collapsed seeds onto near-identical layouts; v0.2 keeps the base model's spread. ² mean horizontal drift of the image centroid relative to the base model's output for the same prompt and seed, in image widths; 0 means the student places its subjects where the base model does. In practice v0.2 follows the prompt and the reference images noticeably more faithfully than v0.1 — fewer dropped or merged instructions, less ghosting of duplicated subjects — and matches the base model's sharpness where the v0.1 LoRA was softer.

The v0.1 artifacts (transformer/ full fine-tune and the r64 LoRA, both 4-step) stay in the repository unchanged for reproducibility; there is no reason to prefer them.

Status: preview, work in progress. v0.2 is a large step up from v0.1 but still falls short of the 40-step base model on complicated image editing (multi-reference composition, face swaps, identity-preserving edits, instructions with several constraints). We will keep updating this repository as the distillation improves.

What's inside

Qwen-Image-2.1-viggle-turbo-v0.2-5step-lora-r256.safetensors   # v0.2 LoRA (rank 256, alpha 256), diffusers key format, bf16  <- use this
peft_v0.2/                                                     # the same v0.2 adapter in peft key format, F32 as trained
scheduler/scheduler_config.json                                # base scheduler config with shift_terminal: null
Qwen-Image-2.1-viggle-turbo-4step-lora-r64.safetensors         # v0.1 LoRA (rank 64), 4-step, kept for reference
peft/                                                          # v0.1 adapter in peft key format, F32
transformer/                                                   # v0.1 full fine-tuned transformer (bf16), 4-step, kept for reference
LICENSE, NOTICE, README.md

Shipped checkpoint: v0.2 is the step-600 EMA student of a LoRA run (rank 256 on the attention, image-MLP, modulation and timestep-embedder projections) that adds SenseFlow-style intra-segment guidance (the student's velocity over each of its segments is regressed on the teacher's) and prompt-enhanced teacher targets to the DMD objective; it was trained on the 4-step schedule and is sampled with the first segment split (5 steps), which we found removes most of the detail loss and ghosting of the plain 4-step rollout. The LoRA is never merged into the transformer — merging into bf16 is lossy, loading it at runtime is exact. Text encoder, VAE and processor are not redistributed; they load from the base repo.

Install

pip install -U torch "transformers>=5.17,<6" accelerate safetensors peft pillow
pip install "git+https://github.com/huggingface/diffusers.git@80c7ed262aeffbeb43ef13ae04baeb9b84515a69"

QwenImage21Pipeline is not in a released diffusers yet, hence the pinned git install. peft is required.

Usage

Load the v0.2 LoRA

import torch
from diffusers import QwenImage21Pipeline, FlowMatchEulerDiscreteScheduler

pipe = QwenImage21Pipeline.from_pretrained("Qwen/Qwen-Image-2.1", dtype=torch.bfloat16)
pipe.load_lora_weights(
    "Viggle/Qwen-Image-2.1-viggle-turbo",
    weight_name="Qwen-Image-2.1-viggle-turbo-v0.2-5step-lora-r256.safetensors",
)
pipe.scheduler = FlowMatchEulerDiscreteScheduler.from_pretrained(
    "Viggle/Qwen-Image-2.1-viggle-turbo", subfolder="scheduler"
)
pipe.to("cuda")

STEPS, SIGMAS = 5, [1.0, 0.875, 0.75, 0.5, 0.25]   # the v0.2 schedule; pass both to every call

# v0.1 full fine-tuned transformer
from diffusers import QwenImage21Transformer2DModel
transformer = QwenImage21Transformer2DModel.from_pretrained(
    "Viggle/Qwen-Image-2.1-viggle-turbo", subfolder="transformer", torch_dtype=torch.bfloat16
)
pipe = QwenImage21Pipeline.from_pretrained("Qwen/Qwen-Image-2.1", transformer=transformer, dtype=torch.bfloat16)
# v0.1 LoRA
pipe.load_lora_weights("Viggle/Qwen-Image-2.1-viggle-turbo", weight_name="Qwen-Image-2.1-viggle-turbo-4step-lora-r64.safetensors")
# both: the shipped scheduler as above, then num_inference_steps=4 and no sigmas= argument

Never stack two students (a LoRA on top of the fine-tuned transformer, or both LoRAs).

Text to image

image = pipe(
    prompt="A studio portrait of an old fisherman mending a net, warm rim light, 85mm.",
    height=1024,
    width=1024,
    num_inference_steps=STEPS,
    sigmas=SIGMAS,
    true_cfg_scale=1.0,                                   # no CFG (also the default)
    generator=torch.Generator("cuda").manual_seed(0),
).images[0]
image.save("out.png")

Image editing (1–3 reference images)

from diffusers.utils import load_image

image = pipe(                                             # same pipe object as above
    prompt="Replace the background with a sunset beach, keep the subject unchanged.",
    image=[load_image("input.png")],                      # list; order fixes <image1>, <image2>, ...
    output_resolution=1024,
    num_inference_steps=STEPS,
    sigmas=SIGMAS,
    true_cfg_scale=1.0,
    generator=torch.Generator("cuda").manual_seed(0),
).images[0]

Rules that matter

• num_inference_steps=5 with sigmas=[1.0, 0.875, 0.75, 0.5, 0.25], true_cfg_scale=1.0, no negative prompt. These are raw sigma nodes: the pipeline applies its resolution-dependent time shift to them exactly as it does to its default nodes, so pass them as written for every resolution. They are the 4-step training nodes linspace(1, 1/4, 4) with the first (highest-noise) segment split in two. num_inference_steps=4 without sigmas= is the training schedule and also works, with slightly less detail and diversity (table above); other step counts and CFG do not help.
• Use the shipped scheduler config (or FlowMatchEulerDiscreteScheduler.from_config(pipe.scheduler.config, shift_terminal=None)). The base config's shift_terminal: 0.02 would wreck the last step.
• Leave the LoRA scale at 1.0 (alpha equals rank).
• Reference-image order determines which image image 1 / image 2 in the prompt refers to. Without height/width, the output aspect ratio follows the last reference. Up to 3 references were used in training.
• Prompt rewriting with the official PE-T2I / PE-I2I rewriters is recommended: v0.2 was distilled against prompt-enhanced teacher targets, and the rewriters help composition and rendered text. The numbers above use them. Raw prompts work too.
• peft users can load peft_v0.2/ directly: pipe.transformer.load_lora_adapter("Viggle/Qwen-Image-2.1-viggle-turbo", subfolder="peft_v0.2", weight_name="adapter_model.safetensors", prefix=None) — same weights, different key names; pick one path, not both.

Resolutions

Training sizes follow the base model's calculate_dimensions(area, aspect_ratio) rounded to a multiple of 32. Text-to-image was trained at 1024² and 2048² area, editing at 1024² and 1536² area (references always encoded at 1024² area). Other sizes work; these are where the training mass sits.

Known limitations

v0.2 closes much of the v0.1 gap but the student still falls short of the base model on complicated editing; text-to-image is close. Specifically:

• Complicated edits degrade: multi-reference composition, face swaps and identity-document edits can still produce duplicated or ghosted figures; "keep everything the same" requests can drift in identity. Much rarer than in v0.1.
• Small or long rendered text can garble more often than with the 40-step base model.
• 2K output is not validated against the teacher.
• Untested: RGBA output, more than 3 reference images, mask-based local editing.
• The two metrics above are our own, computed on 96 / 32 held-out user requests; no standard benchmark is claimed.

License

This model is a derivative work of Qwen-Image-2.1 and is distributed under the Qwen RESEARCH LICENSE AGREEMENT (LICENSE): non-commercial use only — research or evaluation purposes. Commercial use requires a separate licence from the licensor (model-business@notice.qwencloud.com). See NOTICE for the required attribution.

Qwen is licensed under the Qwen RESEARCH LICENSE AGREEMENT, Copyright (c) 2026 Hangzhou Tongyi Laboratory Technology Co., Ltd. All Rights Reserved.

Relative to Qwen/Qwen-Image-2.1 this repository adds two LoRA adapters (v0.2 and v0.1) and a scheduler config with shift_terminal changed from 0.02 to null, and ships a modified transformer (transformer/, the v0.1 full fine-tune); text encoder, VAE and processor are not redistributed.

Distillation method: DMD2 (Yin et al., 2024) / SenseFlow-style distribution matching. Distillation and release by Viggle. Built with Qwen.