Viggle-Animate — ComfyUI conversions
ComfyUI-native conversions of Viggle/Viggle-Animate —
a 33.1 B full finetune of MiniMax-H3's ref2va transformer for character replacement in video:
a driving clip provides motion, camera, background and lighting; one reference still provides the
character. No text encoder — conditioning is a frozen 362-token embedding. The sampler is
DMD2-distilled: 4 steps (3 forward passes).
ComfyUI node pack (required for the conditioning layout): github.com/Saganaki22/ComfyUI-Viggle-Animate-H3
| Example | Video |
|---|---|
| Example 1 | |
| Example 2 | |
| Example 3 | |
| Example 4 |
Model Links
diffusion_models (pick one — pruned is the VRAM-friendly option)
- minimax_h3_ref2va_viggle_pruned_int8_convrot.safetensors (21 GB)
- minimax_h3_ref2va_viggle_int8_convrot.safetensors (47 GB)
- minimax_h3_ref2va_viggle_bf16.safetensors (66.3 GB, max quality)
loras (DMD 4-step accelerator — pick one)
- viggle_animate_dmd_lora_r64.safetensors (0.94 GB, recommended)
- viggle_animate_dmd_lora.safetensors (3.8 GB, full rank)
text_cond
- fixed_embed_fwd_anyframe.safetensors — precomputed text conditioning, load with Load Text Conditioning (Viggle) (no text encoder needed)
vae
- minimax_h3_video_vae_int8_convrot.safetensors (3.17 GB, low VRAM)
- or minimax_h3_video_vae_fp16.safetensors (5.21 GB)
Model Storage Location
📂 ComfyUI/
├── 📂 models/
│ ├── 📂 diffusion_models/
│ │ └── minimax_h3_ref2va_viggle_pruned_int8_convrot.safetensors
│ ├── 📂 loras/
│ │ └── viggle_animate_dmd_lora_r64.safetensors
│ ├── 📂 text_cond/
│ │ └── fixed_embed_fwd_anyframe.safetensors
│ └── 📂 vae/
│ └── minimax_h3_video_vae_int8_convrot.safetensors
Workflow Notes
- Custom nodes required: ComfyUI-Viggle-Animate-H3 (Viggle Animate Conditioning + Load Text Conditioning) and ComfyUI-KJNodes (fast preview).
- Load Video: set
frame_load_capequal to the conditioning node'slength(e.g. 124) andforce_rateto 24. - Output resolution follows the driving video by default; set the conditioning node's
width/heightto override (each axis rounds to 32), or pre-scale the clip with Scale Image to Total Pixels. Tested canvas range: 0.4–0.98 MP. - Sampler: euler, er_sde, exp_heun_2_x0, lcm, simple, normal, beta, bong_tangent, 4-8 steps, cfg 1.0, ModelSamplingMiniMaxH3 shift 3.0.
- Stacks with Comfy Kitchen and block sparse attention patches.
Limitations
- Identity drift on re-entry: when the subject leaves the camera view and re-enters, the re-entry settles toward the driving video's original appearance rather than the reference image. The same applies when the subject moves far from the reference pose or makes abrupt large motions (e.g. a backflip) — the further from the still, the weaker the identity hold.
Base model (VAEs, audio VAE): MiniMaxAI/MiniMax-H3 · Original model & inference code: Viggle/Viggle-Animate
Citation
@misc{viggle2026animate,
title = {Viggle-Animate: Character Replacement in Video from a Single Repainted Frame},
author = {Viggle Research},
year = {2026},
url = {https://huggingface.co/Viggle/Viggle-Animate}
}
License
The weights are a Model Derivative of MiniMax H3 — the MiniMax H3 Community License Agreement applies to them, including these converted/quantized variants. Read it before redistributing or shipping a product on them. Do not run the model on people who have not consented, and label generated output as AI-generated.
Report Issue
- Weights/conversion issues: ComfyUI-Viggle-Animate-H3/issues