VGGT-Diff NVS 480x832 (100K + 6K)
This is the full-resolution independent-view scaling checkpoint obtained by adapting VGGT-Diff's effective-100K half-resolution model for 6,000 additional full-resolution optimizer steps. It takes six posed source images and jointly predicts one or more target views.
Training recipe
The checkpoint was trained in two stages on the DL3DV clean-10K split, with all benchmark-overlapping scenes excluded (9,510 available training scenes):
- Full-model fine-tuning for 100,000 effective optimizer steps at 192 x 336.
- Initialization from that complete half-resolution checkpoint, followed by 6,000 additional full-model fine-tuning steps at 480 x 832.
Both stages use six input views. For each training sample, the number of
supervised target views is sampled independently and uniformly from
{4, 8, 12, 16}. The backbone learning rate is held constant at 5e-6, the
new-module learning rate at 5e-5, and the training seed is 260408500.
Frozen VGGT-Omega features are read from cached bundles, while the VGGT-Diff
DiT is fully fine-tuned.
Configuration
- Resolution: 480 x 832
- Task: independent-view novel view synthesis
- Input: six posed source views and one or more target cameras
- VAE: source and target views are encoded independently
- Geometry: frozen VGGT-Omega features with camera conditioning
- Training data: DL3DV clean-10K (9,510 available scenes)
- Training: full-model fine-tuning
- Release checkpoint: 100K half-resolution steps + 6K full-resolution steps
Table 1 evaluation
The formal evaluation uses independent 6-to-1 inference at 480 x 832, 50
diffusion steps, seed 20260823, CFG 1.0, a 480 x 480 center metric crop,
LPIPS-Alex, DreamSim, and view-weighted aggregation.
| Benchmark | Targets | PSNR | SSIM | LPIPS | DreamSim |
|---|---|---|---|---|---|
| DL3DV | 6,188 | 19.2831 | 0.5685 | 0.1978 | 0.0544 |
| Mip-NeRF 360 | 246 | 17.2170 | 0.3616 | 0.2826 | 0.0795 |
Mip-NeRF 360 is evaluated against the camera-matched FrameCrafter-native ground truth used by the paper protocol. Both rows have zero missing targets.
Usage
Use the nvs architecture mode:
python scripts/infer.py \
--training-mode nvs \
--checkpoint /path/to/vggt-diff.safetensors \
--omega-checkpoint /path/to/vggt_omega_1b_512.pt \
--height 480 --width 832
VGGT-Omega weights and Wan2.1 base-model files are separate dependencies and are not redistributed in this repository.
Checksum
e0cd8798c6247fca0cb4372b6bc87ba561b74e9a0cad4113d1092cb68823eb49 vggt-diff.safetensors
Links
License
Released under the non-commercial VGGT-Diff Research License. Third-party models and datasets remain subject to their own terms.