Track4D-2026
Public model adapters and exact evaluation assets for Track4D, a conditional-video approach to dense 3D tracking. The repository contains separate RTV, NCMV, and Mixture-of-Transformers (MoT) model lines. Their checkpoints are not interchangeable.
Model adapters
| File | Model and intended use |
|---|---|
mot/track4d_mot_wan22_5b_recovery1000.safetensors | MoT recovery1000 adapter for Wan2.2-TI2V-5B; dense RGB + geometry tracking. Contains 1,475 trained tensors (588 Wan, 822 MoT, 62 DPT, 3 XYZ encoder). It is not a standalone base model and contains no optimizer state. |
ncmv/wan21_t2v_14b_512x512x49_lora_r64_step4000.safetensors | NCMV-only Wan2.1-T2V-14B adapter, native 512 × 512 × 49. |
rtv/joint_rtv_ncmv_wan21_i2v_14b_lora_r64_step5000.safetensors | Joint RTV + NCMV Wan2.1-I2V-14B adapter, trained with 49-frame diverse-resolution data. |
rtv/rtv_only_noqi_wan21_i2v_14b_lora_r64_step9000.safetensors | RTV-only/noQI Wan2.1-I2V-14B adapter, trained with 49-frame diverse-resolution data. |
experimental/da3_vdm_vitdpt_step016000/ | Retained Wan LoRA, ViT/DPT probe, and DA3 LoRA for the live-DA3 3D pipeline. |
experimental/dense2d_step013500/ | Retained Wan LoRA and ViT/DPT probe for dense UV tracking plus visibility. |
The MoT recovery1000 adapter SHA256 is
adb6584dee2d640c299dc8b9fc84ae1377ad46ba6a977960c55536dd7ca5bc26 and its
size is 7,129,090,808 bytes. The exact byte sizes and hashes for all released
adapters are in checkpoint_manifest.json.
MoT inference
The public GitHub repository includes the strict model loader, inference protocol, and versioned config. With the separately obtained Wan2.2-TI2V-5B base model, T5 encoder/tokenizer, VAE, and prepared geometry cache installed, the config resolves the MoT adapter from this repository automatically:
python scripts/train/launch_v2v.py \
configs/experiments/infer_mot_recovery1000_worldtrack49.yaml
Set MOT_CHECKPOINT=/path/to/local.safetensors to override the hosted adapter
with a local file. The strict loader verifies the trained topology. Model
design, evaluation protocol, and limitations are documented in
docs/MOT.md.
Wan base models, geometry models, and datasets are obtained separately.
The MoT Omega-DPT component retains its upstream FAIR Noncommercial Research
License; see third_party/
in the code repository. This Hub adapter does not grant rights to redistribute
or use third-party base models beyond their own terms.
RTV/NCMV evaluation assets
The code repository downloads each group selectively with
scripts/download_eval_assets.py:
evaluation/worldtrack_release/: 208 benchmark input/GT NPZ files.evaluation/stream3r_short256_49f/: 200 aligned RGB/geometry NPZ files for the stable RTV replay.evaluation/da3base_step012500_predictions/: exact historical 3D predictions plus evaluator JSON/TXT.evaluation/dense2d_step011500/: five-source metric JSONs and qualitative NPZ/MP4 examples.
For the released WorldTrack RTV scorecard, short256 is both the aligned
membership and the short-side-256 Stream3R scaffold contract. ADT uses
256 × 256; PO, PStudio, and DS/DR preserve landscape aspect and use the
448 × 256 model bucket. Joint RTV+NCMV uses one denoising step; RTV-only uses
50. Exact replay uses base seed 42 plus the global video index over the fixed
ADT, PO, PStudio, DS/DR order.
The joint RTV+NCMV model is a continuation from RTV-only. Its historical score improvement describes the complete two-stage recipe and is not an isolated causal estimate of the NCMV loss. Old experimental 3D step-12.5K and dense-2D step-11.5K model weights were rotated during continued training; saved outputs reproduce their published numbers, while the retained 16K/13.5K weights are not misidentified as those historical metric checkpoints.
Dataset-specific redistribution terms remain separate from the Track4D code license. Historical NPZ files can contain provenance-only source path strings; evaluation uses only their embedded arrays.