LiveAvatar GGUF for audio.cpp
Native GGUF package of Wan-AI/Wan2.2-S2V-14B with the official Quark-Vision/Live-Avatar LoRA merged for audio.cpp. LiveAvatar generates an audio-driven avatar video from a reference image, speech audio, and a text description.
Files
| File | Description |
|---|---|
Wan2.2-S2V-Support-Q4_K_S-F16.gguf | UMT5 text encoder, Wav2Vec2 audio encoder, tokenizer, and embedded audio.cpp model specification. |
Wan2.2-S2V-VAE-F16.gguf | Wan video VAE in F16 with the embedded audio.cpp model specification. |
Wan2.2-S2V-14B-NVFP4-LORA.gguf | LiveAvatar Wan2.2 S2V denoiser with the official LiveAvatar adapter. |
All three files are required.
Low-VRAM mode
LiveAvatar can keep the denoiser transformer blocks in pinned host memory and stage one layer group at a time. This makes 720p generation possible near a 16 GiB VRAM limit, at the cost of additional host-to-device transfers.
The validated 1280x720 run generated 93 frames at 16 FPS (5.81 seconds of video) in 276 seconds wall time (4 minutes 36 seconds) and peaked at 15,993 MiB VRAM on an RTX 5090.
| Control | Value |
|---|---|
liveavatar.denoiser_weight_streaming | true |
denoiser_layerwise | true |
denoiser_layerwise_batch | 16 |
target_cache_blocks | 1 |
vae_cache_f16 | true |
vae_encoder_chunk_size | 4 |
vae_decoder_tile_size | 320 |
Weight streaming is slower than the normal resident-weight path. In a matched 240p comparison, model time increased from 13.33 to 21.73 seconds, or about 1.63x slower. The exact slowdown depends on resolution, hardware, and host memory bandwidth.
Full-duration 720p example
This example uses the official LiveAvatar Cyclops Baker image and speech audio. The official reference image is only 720x400, so this example also shows 1280x720 generation from a lower-resolution image condition. The complete 11.6-second WAV is retained in the generated video.
| Measurement | Result |
|---|---|
| Reference image | 720x400 |
| Reference audio | 11.60 s, 24 kHz mono |
| Output | 1280x720, 16 FPS, 11.60 s |
| Generated frames | 189 before audio-length muxing |
| LiveAvatar clips | 4 |
| Peak VRAM | 16,126 MiB (15.75 GiB) |
| Session time | 543.03 s (9 min 3 s) |
| Full CLI wall time | 557.33 s (9 min 17 s) |
The run used the low-VRAM controls above with denoiser weight streaming, layerwise batch size 16, one target-cache block, F16 VAE cache, VAE encoder chunk size 4, and VAE decoder tile size 320. Measurements were collected with the CUDA debug build on an NVIDIA GeForce RTX 5090.
480p cache trade-off
These two videos use the same Cyclops Baker inputs, prompt, seed, four clips,
and complete 11.6-second audio. Only target_cache_blocks changes. A longer
cache retains more temporal context from preceding video blocks and can improve
continuity across clip boundaries, but it does not directly increase per-frame
resolution or detail.
| Target cache | Peak VRAM | Peak host RAM | Session time | Practical trade-off |
|---|---|---|---|---|
| 2 blocks | 15,311 MiB | 43.33 GiB | 317.92 s | Lower memory and faster; recommended for iteration. |
| 3 blocks | 19,357 MiB | 56.94 GiB | 512.69 s | More temporal history, but nearly exhausts a 64 GB host and uses swap. |
480p is the better starting point for prompt, identity, motion, and cache-window experiments. Iterate at 480p, select the strongest result, and upscale that video afterward instead of paying the 720p generation cost for every attempt.
Quality comparison
Each clip uses the same reference image, audio, prompt, seed 420, four Euler
steps, guidance 0, scheduler shift 3, 416x240 resolution, 16 FPS, and 84
requested frames (81 output frames, 5.06 seconds). The reference uses
SageAttention, memory_saver=true, full target cache, F32 VAE cache, no VAE
tiling, and non-layerwise denoising. Each other cell changes only the parameter
shown below the video.
Inputs: reference speech and reference image.
Prompt: A stout, cheerful dwarf with a magnificent braided beard adorned with metal rings, wearing a heavy leather apron. He is standing in his fiery, cluttered forge, laughing heartily as he explains the mastery of his craft, holding up a glowing hammer. Style of Blizzard Entertainment cinematics, warm, dynamic lighting from the forge.
Measured with the CUDA debug build on an NVIDIA GeForce RTX 5090. Wall time includes model loading and output generation. Peak VRAM is total device memory used during the run.
| Case | Controlled option | Peak VRAM (MiB) | Wall time (s) |
|---|---|---|---|
| Reference | Reference controls | 25,586 | 24.44 |
| FlashAttention | sage_attention=false | 25,990 | 24.90 |
| F16 VAE cache | vae_cache_f16=true | 22,896 | 25.16 |
| VAE decoder tiling | vae_decoder_tile_size=320 | 25,566 | 24.51 |
| Target cache: 1 block | target_cache_blocks=1 | 19,935 | 23.28 |
| Target cache: 4 blocks | target_cache_blocks=4 | 22,761 | 24.40 |
| Layerwise denoising | denoiser_layerwise=true, denoiser_layerwise_batch=16 | 25,333 | 26.62 |
Run
audiocpp_cli \
--task gen \
--family liveavatar \
--model /path/to/LiveAvatar-GGUF \
--backend cuda \
--threads 8 \
--audio /path/to/reference.wav \
--text "A detailed description of the speaker and scene." \
--request-option generation_mode=liveavatar \
--request-option reference_image_path=/path/to/reference.jpg \
--request-option height=240 \
--request-option width=416 \
--out-dir outputs/liveavatar \
--log
The default LiveAvatar configuration uses four Euler steps, scheduler shift 3, guidance scale 0, seed 420, 48 frames per clip, SageAttention, and 16 FPS.
For lower VRAM use, add:
--session-option liveavatar.denoiser_weight_streaming=true \
--request-option denoiser_layerwise=true \
--request-option denoiser_layerwise_batch=16
See the audio.cpp LiveAvatar documentation for resolution, duration, and memory-control options.
License
LiveAvatar and its Wan2.2 base model are released under the Apache License 2.0.
This repository includes the upstream license in LICENSE. The GGUF conversion
is a packaging format for audio.cpp and is not an official upstream release.