Wan2.1-Fun-Reward-LoRAs
Introduction
We explore the Reward Backpropagation technique 1 2 to optimized the generated videos by Wan2.1-Fun for better alignment with human preferences. We provide the following pre-trained models (i.e. LoRAs) along with the training script. You can use these LoRAs to enhance the corresponding base model as a plug-in or train your own reward LoRA.
For more details, please refer to our GitHub repo.
| Name | Base Model | Reward Model | Hugging Face | Description |
|---|---|---|---|---|
| Wan2.1-Fun-1.3B-InP-HPS2.1.safetensors | Wan2.1-Fun-1.3B-InP | HPS v2.1 | 🤗Link | Official HPS v2.1 reward LoRA (rank=128 and network_alpha=64) for Wan2.1-Fun-1.3B-InP. It is trained with a batch size of 8 for 5,000 steps. |
| Wan2.1-Fun-1.3B-InP-MPS.safetensors | Wan2.1-Fun-1.3B-InP | MPS | 🤗Link | Official MPS reward LoRA (rank=128 and network_alpha=64) for Wan2.1-Fun-1.3B-InP. It is trained with a batch size of 8 for 7,500 steps. |
| Wan2.1-Fun-14B-InP-HPS2.1.safetensors | Wan2.1-Fun-14B-InP | HPS v2.1 | 🤗Link | Official HPS v2.1 reward LoRA (rank=128 and network_alpha=64) for Wan2.1-Fun-14B-InP. It is trained with a batch size of 32 for 3,000 steps. |
| Wan2.1-Fun-14B-InP-MPS.safetensors | Wan2.1-Fun-14B-InP | MPS | 🤗Link | Official MPS reward LoRA (rank=128 and network_alpha=64) for Wan2.1-Fun-14B-InP. It is trained with a batch size of 8 for 4,500 steps. |
Demo
Wan2.1-Fun-1.3B-InP
[!NOTE] The above test prompts are from T2V-CompBench and expanded into detailed prompts by Llama-3.3. Videos are generated with HPSv2.1 Reward LoRA weight 0.5 and MPS Reward LoRA weight 0.7.
Wan2.1-Fun-14B-InP
[!NOTE] The above test prompts are from T2V-CompBench and expanded into detailed prompts by Llama-3.3. Videos are generated with HPSv2.1 Reward LoRA weight 0.7 and MPS Reward LoRA weight 0.7.
Quick Start
Set lora_path and lora_weight in examples/wan2.1_fun/predict_t2v.py.
Limitations
- We observe after training to a certain extent, the reward continues to increase, but the quality of the generated videos does not further improve. The model trickly learns some shortcuts (by adding artifacts in the background, i.e., adversarial patches) to increase the reward.
- Currently, there is still a lack of suitable preference models for video generation. Directly using image preference models cannot evaluate preferences along the temporal dimension (such as dynamism and consistency). Further more, We find using image preference models leads to a decrease in the dynamism of generated videos. Although this can be mitigated by computing the reward using only the first frame of the decoded video, the impact still persists.