EdenLive
EdenLive: Real-Time Generative Rendering from Any Visual Reference
-orange){kind=icon}
Project Page · Code · Models
Zehuan Huang1,, Tianxing Xu1,, Chunli Peng1,, Cheng Zhang1,, Yifei Song1,†, Zeyi Zheng1,†, Zhipeng Yu1, Peng Wang1,‡, Yanpei Cao1,✉
* Equal contribution · † Intern at VAST · ‡ Project lead · ✉ Corresponding author
1VAST · Project Eden
Overview
EdenLive renders a proxy or reference video stream, a target first frame, and a text prompt into refined or stylized video. References can come from 2D/3D scenes, existing engines, or recordings. Upstream code maintains interaction state; the model handles visual rendering.
Control is injected through a vision encoder: a post-trained DINOv3 encoder embeds each proxy frame, and those features condition the diffusion transformer.
The project page demonstrates over 24 FPS, minute-scale generation, and 480P/720P output. The 5B model supports single-GPU deployment at 720P.
This repository holds the model weights. Code, installation instructions, and inference/streaming usage live in the EdenLive repository.
Model weights
hf download VAST-AI/EdenLive --local-dir ./models/EdenLive
Everything loads from one directory, models/EdenLive under the code repository root by
default (about 210 GB in total); nothing is pulled from the Hub at run time once
downloaded.
models/EdenLive/
dit-14b/ diffusion_pytorch_model.safetensors, config.json 14B few-step (chunk-AR) DiT
dit-14b-bidir/ diffusion_pytorch_model.safetensors, config.json 14B bidirectional DiT
dit-5b/ diffusion_pytorch_model.safetensors, config.json 5B few-step (chunk-AR) DiT
dit-5b-bidir/ diffusion_pytorch_model.safetensors, config.json 5B bidirectional DiT
vae-14b/ Wan2.1_VAE.pth, config.json
vae-5b/ Wan2.2_VAE.pth, config.json
decoder-14b/ lighttaew2_1.pth TAEHV decoder (real-time only)
decoder-5b/ lighttaew2_2.pth TAEHV decoder (real-time only)
scheduler-14b/ scheduler_config.json offline only
scheduler-5b/ scheduler_config.json offline only
umt5/ text_encoder/, tokenizer/ UMT5 text encoder
clip/ image_encoder/, image_processor/ CLIP, 14B I2V only
vision-encoder/ backbone/, projection.safetensors proxy-DINO encoder
flux-c2f/ Diffusers FLUX tree + c2f_lora.safetensors workbench first frame (real-time only)
To load the weights from another directory:
python -m edenlive.scripts.infer --config ... model.pretrained_dir=/data/EdenLive
The real-time service reads the same tree through MODEL_ROOT in
realtime/deploy.conf.
See the main repository's README
for how each component is wired into the offline and real-time pipelines.
Usage
Each DiT has a matching inference profile in the code repository:
| Mode | Wan2.1-I2V-14B @ 480p | Wan2.2-TI2V-5B @ 720p |
|---|---|---|
Bidirectional (dit-*-bidir/) | wan21_14b_480p_bidir.yaml | wan22_5b_720p_bidir.yaml |
Few-step, chunk-AR (dit-14b/, dit-5b/) | wan21_14b_480p_few_step.yaml | wan22_5b_720p_few_step.yaml |
Few-step inference (chunk-autoregressive sampling, 4 steps per chunk) on the bundled samples:
torchrun --nproc_per_node=2 -m edenlive.scripts.infer \
--config edenlive/configs/wan21_14b_480p_few_step.yaml \
runtime.sequence_parallel.size=2
python -m edenlive.scripts.infer \
--config edenlive/configs/wan22_5b_720p_few_step.yaml
See the Quick start for bidirectional inference, the real-time service, and your own data.
Citation
@misc{edenlive2026,
title = {EdenLive: Real-Time Generative Rendering from Any Visual Reference},
author = {Zehuan Huang and Tianxing Xu and Chunli Peng and Cheng Zhang and Yifei Song and Zeyi Zheng and Zhipeng Yu and Peng Wang and Yanpei Cao},
year = {2026},
url = {https://vast-eden.ai/?p=edenlive}
}
Acknowledgements
Built on Wan2.1 / Wan2.2, 🤗 Diffusers, DINOv3, FLUX.2 [klein] by Black Forest Labs, the LightX2V LightTAE decoders, and LAION CLIP ViT-H/14 trained with OpenCLIP.
License
The weights in this repository are covered by different licenses, per component:
| Component | License | Notes |
|---|---|---|
dit-*, vae-*, umt5/, scheduler-* | Apache 2.0 | Derived from Wan2.1 / Wan2.2 (Apache 2.0) |
decoder-* | Apache 2.0 | LightTAE decoders from lightx2v/Autoencoders (Apache 2.0) |
clip/ | MIT | LAION CLIP ViT-H/14 trained with OpenCLIP; see NOTICE |
vision-encoder/ | DINOv3 License | DINOv3 by Meta and derivatives of it; see NOTICE |
flux-c2f/ | FLUX Non-Commercial License v2.1 | FLUX.2 [klein] 9B by Black Forest Labs plus a LoRA by Vast AI; non-commercial, non-production use only; see NOTICE |
flux-c2f/ is used only to render the real-time workbench's photoreal first frame; the
offline and real-time video pipelines do not load it. Using the workbench therefore falls
under the FLUX Non-Commercial License, while video generation does not depend on it.
Public end-user or production deployment is not automatically permitted by being free of
charge or described as a demo; check the license and obtain a separate authorization from
Black Forest Labs where required.