RAESR

Release package for the paper's single-pass super-resolution models:

• RAESR: frozen DINOv3-L/16 fused latent → released ViT-XL RoPE decoder.
• RAESR-D: frozen DINOv3-S/16+ fused latent → learned frontend (projector) → the same released decoder.

Both models follow the paper protocol: RGB image → bicubic resize to 512×512 → one forward pass → mean-only color matching in the float domain, before PNG quantization: per RGB channel, clamp(sr - mean(sr,H,W) + mean(lr,H,W), 0, 1), where lr is the same 512×512 bicubic-resized tensor the model consumed. The formula is recorded in every run manifest. There is no hidden color alignment, post-processing, normalization, or implicit fallback model.

The released RAESR decoder is the paper's Stage 3, seed 42, step-600 EMA checkpoint:

• Decoder SHA-256: 2e4de5df246172f067e0437f2346d17ca719a8dd12146624df0042983c580cb3

The encoder layers are named 0-indexed in code and manifests (layers 1..23 for RAESR, layers 1..11 for RAESR-D); these are layers 2–24 and 2–12 in the paper's 1-indexed notation.

Repository contents

ema_decoder.pt                  released decoder checkpoint (ViT-XL, 2D RoPE)
weights/raesr_d_frontend.pt     released RAESR-D frontend weights
raesr/                          model runtime (DINOv3 encoders, decoder, RAESR-D frontend)
scripts/                        infer.py, download_artifacts.py, verify_artifacts.py, smoke_test.py, test_infer.py
benchmark/                      run_raesr_benchmark.py, score.py, summarize.py, validate_outputs.py, verify_paper_reproduction.py, tests/
manifests/                      artifact identities, paper reference metrics, runtime provenance
environment/                    pinned runtime and metric requirements
docs/REPRODUCIBILITY.md         exact protocol and validation behavior

1. Create the environment

The environment file installs Python 3.10, PyTorch 2.7.1 and torchvision 0.22.1 (CUDA 12.8 wheels from the official PyTorch index) and the pinned dependencies:

conda env create -f environment/environment.yml
conda activate raesr

See environment/requirements-runtime.txt and environment/requirements-metrics.txt for pinned non-PyTorch packages.

2. Download and verify model weights

Both released checkpoints are fetched from this repository and verified locally. download_artifacts.py writes them to artifacts/ema_decoder.pt and artifacts/weights/raesr_d_frontend.pt:

python scripts/download_artifacts.py --output-dir artifacts
python scripts/verify_artifacts.py --decoder artifacts/ema_decoder.pt --frontend artifacts/weights/raesr_d_frontend.pt

download_artifacts.py downloads from the main revision by default (override with --revision) and always checks the recorded SHA-256.

Both models need the native DINOv3 source checkout and the canonical Meta weights, which are not redistributed:

<DINOV3_ROOT>/hubconf.py
<DINOV3_WEIGHTS>/dinov3_vitl16_pretrain_lvd1689m-8aa4cbdd.pth       (RAESR, DINOv3-L/16)
<DINOV3_WEIGHTS>/dinov3_vits16plus_pretrain_lvd1689m-4057cbaa.pth   (RAESR-D, DINOv3-S/16+)

Obtain them from https://github.com/facebookresearch/dinov3 under Meta's DINOv3 License. Verify the weights you intend to use before inference (--model selects the DINOv3 checkpoint):

python scripts/verify_artifacts.py \
  --decoder artifacts/ema_decoder.pt \
  --frontend artifacts/weights/raesr_d_frontend.pt \
  --model raesr-d \
  --dinov3-root <DINOV3_ROOT> \
  --dinov3-weights <DINOV3_WEIGHTS>

3. Run folder inference

python scripts/infer.py \
  --input-dir <INPUT_IMAGES> \
  --output-dir <OUTPUT_PNGS> \
  --model raesr \
  --decoder artifacts/ema_decoder.pt \
  --dinov3-root <DINOV3_ROOT> \
  --dinov3-weights <DINOV3_WEIGHTS> \
  --device cuda

Pass --model raesr-d --frontend artifacts/weights/raesr_d_frontend.pt to run RAESR-D, with --dinov3-weights pointing to the folder that holds the DINOv3-S/16+ weight. Without --frontend, the frontend is read from weights/raesr_d_frontend.pt in a full clone of this repository. Both models consume the same input protocol and produce 512×512 outputs.

Supported inputs are PNG, JPEG, BMP, TIFF, and WebP. Each input stem must be unique. Outputs use the same stem with .png; existing valid outputs resume by default. Use --overwrite to rerun selected images, or --num-shards N --shard-index I to launch disjoint GPU workers. Each worker writes run_manifest.shard_XX.json with hashes, runtime versions, inputs, outputs, the model choice, and the fixed protocol.

4. Smoke test

The default smoke test validates image, output, and package contracts without downloading assets or loading a GPU model:

python scripts/smoke_test.py

To verify the released checkpoint hash, add --decoder artifacts/ema_decoder.pt. To execute one real forward pass, add --with-model plus the DINO paths.

5. Evaluate on paired benchmark data

Use this layout:

<DATA_ROOT>/
  lsdir/LR/  lsdir/GT/
  realsr/LR/ realsr/GT/
  drealsr/LR/drealsr/GT/
  div2k/LR/  div2k/GT/

Run inference and the paper metric set in one command:

python benchmark/run_raesr_benchmark.py \
  --data-root <DATA_ROOT> \
  --output-root <RESULTS_ROOT> \
  --model raesr \
  --dinov3-root <DINOV3_ROOT> \
  --dinov3-weights <DINOV3_WEIGHTS> \
  --gpu-ids <GPU_ID_1,GPU_ID_2,...> \
  --metric-cache <METRIC_CACHE> \
  --offline-metrics \
  --expected-counts

Pass --model raesr-d --frontend artifacts/weights/raesr_d_frontend.pt for the RAESR-D benchmark. Outputs are written under <RESULTS_ROOT>/sr/<model>/<dataset> and metrics under <RESULTS_ROOT>/metrics/<model>/<dataset>.csv.

--gpu-ids is optional. When supplied, the runner launches one deterministic input shard per listed physical GPU with CUDA_VISIBLE_DEVICES set for that shard, records the assignment in its manifest, waits for every shard, then scores once on the first listed GPU. Without it, inference remains single-process. Each shard log is stored under <RESULTS_ROOT>/logs/<model>/<dataset>/.

--metric-cache is an explicit root holding the pyiqa, Torchvision, CLIP, and Hugging Face metric weights. It sets TORCH_HOME and HUGGINGFACE_HUB_CACHE for the scorer and checks the selected metric assets before any image is scored. Use --offline-metrics with a complete cache to forbid network access.

The scorer writes per-image rows and one MEAN row with: PSNR-Y, SSIM-Y, LPIPS, DISTS, NIQE, MUSIQ, MANIQA-PIPAL, TOPIQ-NR, and CLIP-IQA. It rejects missing pairs, duplicate stems, mismatched image dimensions, metric failures, and non-finite metric values.

Verify the paper rows

manifests/paper_reference_metrics.json (RAESR) and manifests/paper_reference_metrics_raesr_d.json (RAESR-D) contain the paper-protocol four-split means, unweighted macro means, split counts, artifact hashes, the accepted protocol, and an absolute tolerance of 1e-4. Expected paper macro means: RAESR PSNR 24.0825 / SSIM 0.6269; RAESR-D PSNR 23.39 / SSIM 0.6245. After a four-split run, verify every MEAN row and the macro result:

python benchmark/verify_paper_reproduction.py \
  --metrics-root <RESULTS_ROOT>/metrics/raesr \
  --benchmark-provenance <RESULTS_ROOT>/raesr_benchmark.provenance.json \
  --out <RESULTS_ROOT>/paper_reproduction.json

For RAESR-D, point --metrics-root at <RESULTS_ROOT>/metrics/raesr-d and add --reference manifests/paper_reference_metrics_raesr_d.json.

The command checks the benchmark provenance (artifact hashes, per-shard manifests, output inventories, counts, score sidecars) and then compares every per-dataset and macro metric against the reference manifest. It writes every observed/reference/delta triplet and exits nonzero for a provenance, schema, count, finite-value, or tolerance failure.

Reproducibility files

• manifests/model_artifacts.json: artifact identities (decoder, RAESR-D frontend, DINOv3 checkpoints) and the architecture contract.
• manifests/paper_reference_metrics.json / manifests/paper_reference_metrics_raesr_d.json: paper-protocol reference means, protocol, and tolerance.
• manifests/runtime_provenance.json: source hashes of the runtime and the only changes made for this package.
• docs/REPRODUCIBILITY.md: exact protocol, expected outputs, and validation behavior.

License and attribution

This package is distributed under CC BY-NC 4.0 for non-commercial research use (see LICENSE). Several runtime files are adapted from RAEv2 (CC BY-NC 4.0); THIRD_PARTY_NOTICES.md gives the attribution and lists the changes. The DINOv3 source and weights, including DINOv3-S/16+ used by RAESR-D, are not redistributed; obtain them from https://github.com/facebookresearch/dinov3 under Meta's DINOv3 License.