ReFlowSET

SAR-to-EO image translation with a conditional flow-matching transformer trained from scratch inside a frozen high-fidelity autoencoder latent space.

• Paper: https://arxiv.org/abs/2609.00968
• Code: https://github.com/KAIST-VICLab/ReFlowSET
• Project page: https://kaist-viclab.github.io/ReFlowSET_site/
• Comparison-method weights: baselines/ in this repository — all fifteen prior methods, both datasets

Checkpoints

Each subfolder is a complete diffusers pipeline: transformer/, vae/, scheduler/ and model_index.json. All arms share the same frozen autoencoder; per dataset they differ only in resolution, batch size and step count. SAR2Opt stops at 20,000 steps to hold a comparable sample budget on a 1,450-image training set. The two versions differ in how the EO and SAR streams are merged after the double-stream blocks (see Architecture); ReFlowSET (Channel) is also trained with a variant of the REPA alignment loss (configs on GitHub).

The published weights are the EMA parameters. The training-only REPA projector is not included.

Usage

import torch
from PIL import Image
from diffusers import DiffusionPipeline
from huggingface_hub import snapshot_download

# Every arm lives in this one repository, one per subfolder. `DiffusionPipeline`
# has no `subfolder` argument, so fetch the arm and load it as a local pipeline.
ARM = "qxs-saropt"   # or "sar2opt", "qxs-saropt-channel", "sar2opt-channel"
root = snapshot_download("JeonghyeokDo/ReFlowSET", allow_patterns=[f"{ARM}/*"])
pipe = DiffusionPipeline.from_pretrained(
    f"{root}/{ARM}", custom_pipeline=f"{root}/{ARM}", torch_dtype=torch.float32,
).to("cuda")

sar = Image.open("sar.png")                       # 1-channel SAR, 8-bit PNG
eo = pipe(sar, num_inference_steps=50, guidance_scale=1.5,
          generator=torch.Generator("cuda").manual_seed(2024)).images[0]
eo.save("eo.png")

custom_pipeline points at the same directory because the pipeline, transformer, autoencoder and scheduler classes ship with the checkpoint rather than living in diffusers. The classes are also on GitHub under src/reflowset/.

Sampling settings are part of the reported result, not free knobs. The paper's main table is NFE 50 with guidance scale 1.5. NFE 4 samples 11× faster at 256² (163 ms vs 1824 ms) and 13× faster at 512² (371 ms vs 4807 ms), batch 1 on one B200, and trades distribution metrics against pixel metrics; do not mix the two in one comparison.

The SAR input is read without a colour conversion, collapsed to one channel, center-cropped (never resized), scaled by x / 127.5 − 1, replicated to three channels, and encoded by the same frozen autoencoder that defines the EO latent space. The pipeline does all of this; feed it the raw PNG.

Results

Scored on the same test items as fifteen prior methods that we retrained under one protocol, by a single evaluator.

The full table, with every comparison method's weights and licence, is in MODEL_ZOO.md.

These numbers are not comparable with the ones printed in the source papers. Splits, resolutions and evaluator conventions differ. In particular LPIPS has two conventions in this literature that differ by ~0.05: we feed x*2−1 to the LPIPS network, while several released evaluators feed [0,1] with normalize=False and obtain a systematically lower number.

Architecture

A DiT with hidden size 1024 and depth 24 — eight double-stream blocks that give the EO and SAR streams their own projections and joint attention, then sixteen single-stream blocks — with 16 heads of dimension 64 and 2-D RoPE over axes (32, 32). ReFlowSET (Token) concatenates the two streams on the token axis before the single-stream blocks; ReFlowSET (Channel) concatenates them on the channel axis and projects back to width 1024, keeping one token per position.

There is no separate SAR encoder: the SAR image goes through the same frozen autoencoder as the EO image. Training defines a linear bridge z_t = (1−t)·ε + t·z_e and regresses the velocity u* = z_e − ε conditioned on the SAR latent; sampling starts from N(0, I) and integrates t: 0 → 1 with an explicit Euler step. Classifier-free guidance was trained by zeroing the SAR condition on 10 % of rows.

Training data

QXS-SAROPT and SAR2Opt only, one dataset per arm, trained from scratch. No pretraining corpus. Neither dataset is redistributed. QXS-SAROPT requires citing arXiv:2103.08259 for research use.

Comparison methods, in this same repository

baselines/ holds the fifteen prior methods of the paper's main table, retrained by us on the same splits and scored by the same evaluator — thirty checkpoints, both datasets, each with its own card, its measured row, its upstream repository and its licence. Start at baselines/README.md.

from diffusers import UNet2DConditionModel
unet = UNet2DConditionModel.from_pretrained(
    "JeonghyeokDo/ReFlowSET", subfolder="baselines/qxs-saropt/cdiffset")

Licence and provenance

This repository is mixed-licence, so the Hub tag is other. ReFlowSET's own weights are CC BY-NC 4.0 and the code is Apache-2.0. Everything under baselines/ is a third-party method carrying its own terms — MIT, BSD-3-Clause, Apache-2.0, CreativeML-OpenRAIL-M, CC BY-NC-SA 4.0, S-Lab 1.0, and four with no upstream licence file at all. The per-method cards state each one, and the texts that must travel are in baselines/licenses/. Check the method you intend to use; the repo-level tag is not a substitute.

The frozen autoencoder bundled in vae/ is the Apache-2.0 autoencoder from black-forest-labs/FLUX.2-klein-base-4B, re-serialised to the upstream layout and cast to bfloat16 — tensors paired by value, not by an assumed rename table, and bit-identical to that source through a full encode/decode.

It is a substitution, and here is exactly what was substituted. All arms were trained and evaluated with the FLUX.2-dev serialisation of the same network; that file is under the FLUX Non-Commercial License, whose §4(a)(iii) forbids "research and development related to surveillance" and whose §1(a) makes the restriction inherit permanently. The two serialisations are the same autoencoder — 250 of 251 tensors pair by value, worst absolute deviation 7.8e-03 (bfloat16 rounding) — and swapping the Apache file into the released checkpoints changes QXS-SAROPT PSNR by less than 0.004 dB in absolute value. Four independent measurements at different guidance scales and sample sets land between −0.004 and +0.002 dB, so the sign is not resolved and only the magnitude is meaningful. Changing only the evaluation seed moves the same number by +0.395 dB. Nothing reported here changes.

Apache-2.0 §6 withholds trademark rights. This model is named ReFlowSET; it is not a FLUX product and is not endorsed by Black Forest Labs.

Why this autoencoder. A latent generator cannot beat its codec's round trip, so the codec is a ceiling on every row of a latent-model comparison. vae_audit/ measures that ceiling for six autoencoders — SD2.1, SDXL, SD3.0, SD3.5, FLUX.1 and FLUX.2 — on four SAR/EO benchmarks, EO and SAR scored separately. It ships as code with download links and licence terms; no imagery and no third-party autoencoder weights are redistributed.

Training used a frozen DINOv3 ViT-L/16 (LVD-1689M) as a representation- alignment teacher, acknowledged here as the DINOv3 License §1(b)(ii) requires. The teacher is not loaded at inference and no DINOv3 weights are redistributed; obtain them from Meta's release under its own terms if you intend to retrain.

See LICENSE-WEIGHTS.md for the full reasoning, including one open question about the datasets' optical imagery that we flag rather than resolve.

Citation

@article{do2026reflowset,
  title   = {ReFlowSET: Representation-Aligned Latent Flow Matching for SAR-to-EO Image Translation},
  author  = {Do, Jeonghyeok and Lee, Seungchul and Kim, Munchurl},
  journal = {arXiv preprint arXiv:2609.00968},
  year    = {2026}
}

Our earlier SAR-to-EO work, which ReFlowSET builds on and compares against:

@article{do2026cdiffset,
  title   = {C-DiffSET: Leveraging Latent Diffusion for SAR-to-EO Image Translation with Confidence-Guided Reliable Object Generation},
  author  = {Do, Jeonghyeok and Lee, Jaehyup and Lee, Seungchul and Kim, Munchurl},
  journal = {IEEE Transactions on Circuits and Systems for Video Technology},
  year    = {2026}
}