SwiftVR

SwiftVR is the first generative video restoration model to reach real-time 1080p streaming on a consumer-grade GPU (≈26 FPS on a single RTX 5090), sustains 31 FPS at QHD (2560×1440) and 14 FPS at 4K (3840×2160) on a single H100, and streams at resolutions where every compared diffusion-based VR baseline runs out of memory.

SwiftVR is a streaming one-step generative video restoration (VR) framework presented in SwiftVR: Real-Time One-Step Generative Video Restoration.

Updates

• [2026/06] Release the inference code and pretrained weights 🎉

Community Works

• LightX2V brings faster inference and lower GPU memory usage to SwiftVR (1.91× speedup and 65.71% lower peak GPU memory per request on a single H100). It also supports multi-GPU acceleration. Ready-to-use scripts cover image and video super-resolution, offline inference, and API serving. Get started with LightX2V →

✨ Highlights

• Mask-free shifted-window self-attention (MFSWA). Each spatial window is pre-gathered into a dense tensor, so every attention call reduces to a single standard scaled-dot-product (SDPA) call — no attention mask, cyclic shift, or padding ever enters the graph. This gives a 1.62× throughput gain over its full-attention teacher at essentially identical quality, with no dedicated sparse kernel.
• Restoration-aware Autoencoder (ReAE). A lightweight encoder–decoder jointly fine-tuned with the DiT in pixel space removes the heavy-3D-VAE / tiled-decoding bottleneck.
• Causal chunk-wise streaming. A minimal causal protocol (no rolling KV cache, no overlapped DiT inference) bounds the temporal axis, confining the residual (\mathcal{O}(N^2)) cost to the spatial axes.

📊 Results

Efficiency at 2560×1440 (single H100, causal streaming, 24 frames)

At 3840×2160, every compared diffusion-based VR baseline OOMs on a single H100; SwiftVR sustains 14 FPS.

Qualitative comparison

🛠 Installation

git clone https://github.com/H-oliday/SwiftVR.git
cd SwiftVR

conda create -n swiftvr python=3.10 -y
conda activate swiftvr

# Install PyTorch matching your CUDA toolkit first, e.g. CUDA 12.4:
pip install torch==2.10.0 torchvision==0.25.0 --index-url https://download.pytorch.org/whl/cu124

# Install SwiftVR (editable) and its dependencies:
pip install -e .

• Server: single H100-80G reproduces the QHD/4K numbers above.
• Consumer: single RTX 5090 reaches ≈26 FPS at 1080p with the same checkpoint (default PyTorch SDPA path, bfloat16, causal chunk protocol).
• No hardware-specific retraining or kernel rewrite is required on any platform.

🗂 Model Zoo

huggingface-cli download H-oliday/SwiftVR --local-dir checkpoints/

Expected checkpoint layout (the directory passed to from_pretrained):

checkpoints/
├── reae.safetensors            # Restoration-aware Autoencoder weights
├── prompt_embedding.safetensors# precomputed empty-prompt text embedding (key: "prompt_emb")
└── transformer/                # diffusers-format DiT
    ├── config.json
    └── diffusion_pytorch_model.safetensors

🚀 Quick Start

Python API

from swiftvr import SwiftVRPipeline

pipe = SwiftVRPipeline.from_pretrained("H-oliday/SwiftVR").to("cuda", dtype="bfloat16")

pipe.restore_video("low_quality.mp4", "restored.mp4", upscale=4)

restore_video also accepts an image folder as input and can write a PNG sequence with png_save=True.

Tunable knobs include:

• clip_len: middle chunk size, multiple of 4
• dit_overlap: overlap for DiT inference
• fps: output video frame rate
• quality: 0–100, mapped to x265 CRF
• queue_size: pipeline queue size

Streaming (causal, chunk by chunk, no future frames)

Causal, chunk-by-chunk restoration without future frames.

session = pipe.stream(clip_len=24, resolution=(1920, 1080))

for lq_chunk in read_chunks("low_quality.mp4", n=24):   # lq_chunk: [T, H, W, 3] uint8
    hq = session.step(lq_chunk)        # [1, T', 3, H', W'] in [0, 1], or None if buffered
    if hq is not None:
        write(hq)

tail = session.flush()                 # flush the final buffered frames

Command line

python scripts/inference.py \
  --input low_quality.mp4 \
  --output restored.mp4 \
  --checkpoint checkpoints/ \
  --upscale 4 \
  --clip-len 24 \
  --dtype bfloat16 \

Use --png to write a PNG sequence.

🎬 More Visual Results

Full-length restored clips (low-quality input → SwiftVR, played back to back).

🙏 Acknowledgements

SwiftVR builds on Wan2.2-TI2V-5B, the lightweight autoencoder TAEHV, and the RealBasicVSR degradation pipeline. We thank the authors of DOVE, SeedVR2, and FlashVSR for releasing strong baselines, and the UltraVideo team for the training corpus.

📜 License

The model weights in this repository are licensed under the SwiftVR Research License. See the LICENSE file.

Permitted without separate authorization

• Non-commercial academic research;
• Personal non-commercial research;
• Teaching and education;
• Reproduction of academic results;
• Internal evaluation for determining whether to request a commercial license.

Prior written authorization required

• Use in a commercial product or service;
• Production deployment;
• Customer-facing use;
• Paid API, SaaS, cloud, or hosted services;
• Paid consulting or contract work;
• Processing videos on behalf of customers;
• Commercial redistribution of the model or modified versions;
• Use primarily intended to generate revenue or commercial advantage.

Repository access, model download, email acknowledgement, or informal communication does not constitute commercial authorization.

For commercial licensing inquiries, contact:

• Email: kakibluee@gmail.com or chxy95@gmail.com

Source code

The corresponding SwiftVR source code is separately licensed under the Apache License 2.0 in the GitHub repository.

Historical releases

Historical model versions previously released under the Apache License 2.0 remain subject to that license. This license does not retroactively revoke rights validly granted for those historical versions.

📜 Citation

@article{yan2026swiftvr,
  title={SwiftVR: Real-Time One-Step Generative Video Restoration},
  author={Yan, Jiaqi and Chen, Xiangyu and Zhong, Xinlin and Huang, Haibin and Zhang, Chi and Liu, Jie and Zhou, Jiantao and Li, Xuelong},
  journal={arXiv preprint arXiv:2606.09516},
  year={2026}
}

Contact

If you have any questions, feel free to reach out:

• Email: kakibluee@gmail.com or chxy95@gmail.com