VSQA preview 1.3B — DMD step 1000
Preview checkpoint. This is the student of a completed 1000-step DMD run, exported at its final step (1000). It is published for early inspection.
What it is
A few-step (3-step DMD) student of Wan2.1-T2V-1.3B trained with video sparse attention (VSA) and NVFP4 quantization-aware training (QAT) of both attention and linear layers (VSQA project).
Training lineage (all at 61 frames, 896×448, global batch 16, NVFP4 attention and linear QAT):
- Dense NVFP4 per-layer reconstruction (groups of 5 blocks, teacher inputs), from the original Wan2.1-T2V-1.3B weights, LR 1e-6; the step-500 checkpoint is used.
- Sparse ODE initialization with an auxiliary reconstruction loss (endpoint weight 0.5, attention and MLP reconstruction weights 0.25 each), VSA C128 cube (4, 4, 8) at sparsity 0.9, 3-step schedule, LR 1e-5; the step-500 checkpoint is used.
- DMD distillation, 1000 steps (this checkpoint), student LR 2e-6, critic LR 4e-7, generator updated every 5 iterations, no EMA, real-score guidance 3.5. Teacher and critic are the original dense Wan2.1-T2V-1.3B.
Sampling contract
- 3 denoising steps at timesteps
[1000.0, 941.1763916015625, 800.0], no additional timestep warp. flow_shift = 8.0, guidance scale 1.0 (distilled), 61 frames at 896×448.- Attention backend used in training:
VSA_QAT_TRAIN_C128(sparsity 0.9); linear quantizationnvfp4_qat_train. Reproducing the trained numerics needs the FastVideo VSA/QAT kernels; running the weights with plain dense BF16 attention and linears is a different (untested) configuration.
Files
Diffusers layout. transformer/ holds the exported student; text_encoder/, tokenizer/, vae/ and
scheduler/ are unchanged from the base model. metadata.json (normalized training contract),
artifact_manifest.json, resolved.yaml and source.yaml record provenance; the export passed a strict
reload and its payload digests match artifact_manifest.json.
License follows the base model (Apache-2.0).