Installation
Install Diffusers and additional requirements:
uv pip install --torch-backend=auto git+https://github.com/huggingface/diffusers torch torchvision google-genai boltons ujson sentencepiece accelerate transformers
Usage
Write your edit instruction in plain text and convert it, together with the source image, into a structured VGL JSON prompt with briaai/FIBO-edit-gemini-prompt-to-JSON. This block calls Gemini and supports single references, multiple references, and masks. Set its API key first:
export GEMINI_API_KEY="your-api-key"
import torch
from diffusers import BriaFiboEditPipeline
from diffusers.modular_pipelines import ModularPipelineBlocks
from PIL import Image
vlm_pipe = ModularPipelineBlocks.from_pretrained(
"briaai/FIBO-edit-gemini-prompt-to-JSON", trust_remote_code=True
)
vlm_pipe = vlm_pipe.init_pipeline()
# Load the turbo (4-step distilled) pipeline
pipe = BriaFiboEditPipeline.from_pretrained(
"briaai/Fibo-Edit-1.5-turbo",
torch_dtype=torch.bfloat16,
)
pipe.to("cuda")
# Optional: compile the repeated transformer blocks for faster inference.
# pipe.transformer.compile_repeated_blocks()
# Your source image
image = Image.open("photo.jpg")
# Convert the edit instruction to a structured VGL JSON prompt
json_prompt = vlm_pipe(image=image, prompt="make the sky dramatic and stormy").values["json_prompt"]
result = pipe(
image=image,
prompt=json_prompt,
num_inference_steps=4, # 4 steps only — this is a distilled model
guidance_scale=1.0, # CFG 1.0 — no classifier-free guidance needed
).images[0]
result.save("edited.png")
Multiple references
BriaFiboEditPipeline also accepts a list of reference images (multiple references, not a batch). Keep the distilled 4-step / guidance 1.0 defaults:
references = [Image.open("ref-1.png").convert("RGB"), Image.open("ref-2.png").convert("RGB")]
json_prompt = vlm_pipe(
image=references, prompt="place the subject from both references on a beach"
).values["json_prompt"]
result = pipe(
image=references,
prompt=json_prompt,
num_inference_steps=4,
guidance_scale=1.0,
).images[0]
result.save("multi_ref.png")
Architecture
| Component | Details |
|---|---|
| Pipeline | BriaFiboEditPipeline |
| Transformer | BriaFiboTransformer2DModel — 8B parameters (24 heads × 128 dim, 8 joint + 38 single layers) |
| VAE | AutoencoderKLWan — Wan 2.2 VAE (48 latent channels, 16× spatial downscale) |
| Text Encoder | SmolLM3ForCausalLM (bfloat16) |
| Tokenizer | SmolLM PreTrainedTokenizerFast |
| Scheduler | FlowMatchEulerDiscreteScheduler (shift=3.0, dynamic shifting) |
Provenance
- Transformer: DMD student checkpoint (step 1500), distilled from Fibo-Edit 1.5 teacher with character LoRA merged at scale 0.9.
- VAE: Wan 2.2 VAE (shared with the Fibo-Edit family).
- Text Encoder / Tokenizer: SmolLM3 (shared with the Fibo-Edit family).
- Scheduler: FlowMatchEulerDiscreteScheduler (same config as Fibo-Edit).
More Examples
Citation
@article{gutflaish2025generating,
title={Generating an Image From 1,000 Words: Enhancing Text-to-Image With Structured Captions},
author={Gutflaish, Eyal and Kachlon, Eliran and Zisman, Hezi and Hacham, Tal and Sarid, Nimrod and Visheratin, Alexander and Huberman, Saar and Davidi, Gal and Bukchin, Guy and Goldberg, Kfir and others},
journal={arXiv preprint arXiv:2511.06876},
year={2025}
}