Stream-DiffVSR: Low-Latency Streamable Video Super-Resolution via Auto-Regressive Diffusion
Stream-DiffVSR is a causally conditioned diffusion framework designed for efficient online Video Super-Resolution (VSR). It operates strictly on past frames to maintain low latency, making it suitable for real-time deployment.
[Paper] [Project Page] [GitHub]
Description
Diffusion-based VSR methods often struggle with latency due to multi-step denoising and reliance on future frames. Stream-DiffVSR addresses this with:
- Causal Conditioning: Operates only on past frames for online processing.
- Four-step Distilled Denoiser: Enables fast inference without sacrificing quality.
- Auto-regressive Temporal Guidance (ARTG): Injects motion-aligned cues during denoising.
- Lightweight Temporal Decoder: Enhances temporal coherence and fine details.
Stream-DiffVSR can process 720p frames in 0.328 seconds on an RTX 4090, achieving significant latency reductions compared to prior diffusion-based VSR methods.
Usage
Installation
git clone https://github.com/jamichss/Stream-DiffVSR.git
cd Stream-DiffVSR
conda env create -f requirements.yml
conda activate stream-diffvsr
Inference
You can run inference using the following command. The script will automatically fetch the necessary weights from this repository.
python inference.py \
--model_id 'Jamichsu/Stream-DiffVSR' \
--out_path 'YOUR_OUTPUT_PATH' \
--in_path 'YOUR_INPUT_PATH' \
--num_inference_steps 4
The expected file structure for the inference input data is as follows:
YOUR_INPUT_PATH/
├── seq1/
│ ├── frame_0001.png
│ ├── frame_0002.png
│ └── ...
├── seq2/
│ ├── frame_0001.png
│ ├── frame_0002.png
│ └── ...
For NVIDIA TensorRT acceleration:
python inference.py \
--model_id 'Jamichsu/Stream-DiffVSR' \
--out_path 'YOUR_OUTPUT_PATH' \
--in_path 'YOUR_INPUT_PATH' \
--num_inference_steps 4 \
--enable_tensorrt \
--image_height <YOUR_TARGET_HEIGHT> \
--image_width <YOUR_TARGET_WIDTH>
Note
The provided checkpoint is a toy / proof-of-concept model trained on a limited amount of data. As a result, it does not yet cover the full diversity of real-world videos.
This checkpoint is mainly intended to demonstrate the overall pipeline and low-latency feasibility, rather than to deliver production-level upscaling quality.
Artifacts and inconsistent visual quality are therefore expected at this stage.
Citation
If you find this work useful, please cite:
@article{shiu2025stream,
title={Stream-DiffVSR: Low-Latency Streamable Video Super-Resolution via Auto-Regressive Diffusion},
author={Shiu, Hau-Shiang and Lin, Chin-Yang and Wang, Zhixiang and Hsiao, Chi-Wei and Yu, Po-Fan and Chen, Yu-Chih and Liu, Yu-Lun},
journal={arXiv preprint arXiv:2512.23709},
year={2025}
}