VSQA preview 14B — DMD step 100
Preview / intermediate checkpoint. Not evaluated. This is the student of an in-progress 1000-step DMD run, exported at step 100. It is published for early inspection only; later checkpoints of the same run may replace it.
What it is
A few-step (3-step DMD) student of Wan2.1-T2V-14B trained with video sparse attention (VSA) and NVFP4 quantization-aware training (QAT) of both attention and linear layers (VSQA project).
Training lineage (all at 77 frames, 1280×768, global batch 16):
- Dense NVFP4 per-layer reconstruction, 250 steps, from the original Wan2.1-T2V-14B weights.
- Plain ODE initialization (endpoint regression, no reconstruction loss), 500 steps, VSA C256 cube (4, 8, 8) at sparsity 0.9, NVFP4 attention and linear QAT.
- DMD distillation (this checkpoint: step 100 of 1000), student LR 2e-6, critic LR 4e-7, generator updated every iteration, no EMA. Teacher and critic are the original dense Wan2.1-T2V-14B.
Sampling contract
- 3 denoising steps at timesteps
[1000.0, 941.1763916015625, 800.0], no additional timestep warp. flow_shift = 8.0, guidance scale 1.0 (distilled), 77 frames at 1280×768.- Attention backend used in training:
VSA_QAT_TRAIN_C256(sparsity 0.9); linear quantizationnvfp4_qat_train. Reproducing the trained numerics needs the FastVideo VSA/QAT kernels; running the weights with plain dense BF16 attention and linears is a different (untested) configuration.
Files
Diffusers layout. transformer/ holds the exported student; text_encoder/, tokenizer/, vae/ and
scheduler/ are unchanged from the base model. metadata.json (normalized training contract),
artifact_manifest.json, resolved.yaml, source.yaml and export-verification.json record provenance
and the strict-reload export verification.
License follows the base model (Apache-2.0).