FAST: Flexibly Controllable Arbitrary Style Transfer via Latent Diffusion models

Hanzhang Wang, Haoran Wang, Zhongrui Yu, Mingming Sun, Junjun Jiang, Xianming Liu, Deming Zhai.

ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM), 2025.

Paper · arXiv · Project Homepage · GitHub

This repository provides the FAST image/video checkpoints, pretrained dependencies, and annotator implementations and weights.

Contents

checkpoint/
  image_model/                 # FAST image UNet and adapters
  video_model/                 # FAST video UNet and adapters
pretrained_models/
  stable-diffusion-v1-5/        # Diffusers FP32 .bin layout
  dpt-hybrid-midas/             # Depth estimator
  hed-network.pth              # HED edge estimator
  150_16_swin_l_oneformer_coco_100ep.pth
model/annotator/
  hed/                        # HED implementation
  canny/                      # Canny implementation
  pidinet/                    # PiDiNet implementation
  oneformer/                  # OneFormer implementation and configurations
  uniformer/                  # Uniformer implementation and configurations
  ckpts/                      # OneFormer ADE20K and Uniformer weights
  ...

This repository preserves the local directory layout expected by the FAST code. Model files are regular files rather than links to the original author's Hugging Face cache. It contains 659 asset files, approximately 19.82 GiB in total, including the 593 files in model/annotator/ (approximately 1.08 GiB). Datasets are not included.

Download and use

Install the Hugging Face CLI, then run the following commands from the FAST code directory:

hf download wd1511/fast-ldm --include 'checkpoint/**' --local-dir .
hf download wd1511/fast-ldm --include 'pretrained_models/**' --local-dir .
hf download wd1511/fast-ldm --include 'model/annotator/**' --local-dir .
python -m test_sh.test_image --limit 1
python -m test_sh.test_video --limit 1

These commands use the prepared FAST source release with test_sh/test_image.py and test_sh/test_video.py; publication of that code is separate from this model upload. The commands above place model/annotator/ in the expected local directory. Test inputs must be prepared separately as described in docs/ASSETS.md in that source release. Use the image and video checkpoints with their corresponding custom FAST pipelines; these are not drop-in replacements for a standard Stable Diffusion UNet.

FAST checkpoints include their VGG feature extractor parameters. The provided inference scripts load those parameters from the checkpoint without separately downloading VGG.

Upstream components

The pretrained_models/ directory contains dependencies used by FAST, rather than models newly trained for this release: Stable Diffusion v1.5, DPT Hybrid MiDaS, HED and OneFormer COCO. The bundled model/annotator/ directory includes the corresponding third-party implementations, configuration files, license files present in the source package, and additional OneFormer/Uniformer weights. Their original authorship and applicable upstream licenses are retained; inclusion here does not relicense these components.

Citation

@article{wang2025fast,
  title={FAST: Flexibly Controllable Arbitrary Style Transfer via Latent Diffusion models},
  author={Wang, Hanzhang and Wang, Haoran and Yu, Zhongrui and Sun, Mingming and Jiang, Junjun and Liu, Xianming and Zhai, Deming},
  journal={ACM Transactions on Multimedia Computing, Communications and Applications},
  publisher={ACM New York, NY},
  year={2025}
}