TrackWan 1.3B I2V — Motion-Track Teacher (synth stage-2)
A motion-track-conditioned image-to-video model based on Wan2.1 1.3B (Fun-InP). On top of the standard I2V conditioning it takes a set of sparse point tracks (a small number of pixel trajectories with per-frame visibility) and generates a video whose motion follows those tracks. It is the "teacher" model in a MotionStream-style track-conditioning recipe.
This repository is a complete, self-contained diffusers pipeline — transformer, VAE, T5 text encoder, CLIP image encoder, tokenizer, scheduler — so it loads directly with no external weights.
Checkpoint
- Final checkpoint (step 600) of the
synth_stage2_paperLRrun. - Stage-2 fine-tune (lr 1e-6, global batch size 128, 600 steps) on a combined synthetic track dataset, with stochastic track/motion dropping, starting from the stage-1 OpenVid-1M teacher.
- 480×832, 121 frames, 24 fps,
flow_shift = 6.
Architecture
Standard Wan2.1 1.3B DiT (30 layers, hidden 1536) with a WanTrack head:
| field | value |
|---|---|
in_channels | 52 (16 latent + 20 I2V/mask + 16 track channels) |
out_channels | 16 |
track_config.id_dim | 64 (sinusoidal track-id embedding) |
track_config.track_channels | 16 |
track_config.vae_spatial_compression | 8 |
track_config.vae_temporal_compression | 4 |
The track pathway is track_encoder.{temporal_conv, proj} plus the extra
patch_embedding input channels [:, 36:] (the "track slot").
Usage (FastVideo)
The model is a diffusers-format directory and loads directly by path in FastVideo's WanTrack inference / training stack. VAE, T5, and CLIP are the standard Wan2.1 components and are bundled here.
Notes
- VAE / T5 / CLIP are identical to the Wan2.1 base components; they are included so the pipeline is directly loadable.
- Base model: Wan2.1 1.3B (Fun-InP variant).