TalkingAvatar (TA2.0): Audio-Driven Talking Avatar Generation
Reference Image + Speech Audio → Expressive Avatar Video by Neosapience
TalkingAvatar generates expressive, lip-synchronized avatar videos from a reference image, speech audio, and an optional text prompt. Model weights are distributed as sharded safetensors with an integrity manifest and runtime configuration.
Source Code | Training Dataset | Neosapience | Typecast
Training Dataset
TalkingAvatar was trained using the TA2.0 Animation Dataset. See the dataset repository for its documentation, structure, and access details.
Quick Start
Clone the source repository and create its uv environment:
git clone https://github.com/neosapience/TA2.0.git
cd TA2.0
uv venv --system-site-packages .venv
source .venv/bin/activate
uv pip install --no-deps -r requirements.lock
Generate a video with the top-level inference script:
CUDA_VISIBLE_DEVICES=0 uv run --no-sync python inference.py \
--image examples/reference.png \
--audio examples/audio.wav \
--prompt "The person is speaking moderately." \
--save_file outputs/result.mp4
The script downloads the TalkingAvatar checkpoint and resolves its required
video and audio dependencies through the Hugging Face cache.
Run python inference.py --help for all options.
Inference Profile
| Property | Value |
|---|---|
| Resolution | 480 x 832 |
| Frame rate | 25 fps |
| Duration | Derived from the input audio |
| Denoising steps | 30 |
| Text guidance | 5.0 |
| Audio guidance | 3.0 |
The runtime automatically chooses the shortest model-valid frame count that
covers the complete input audio. --frame_num remains available as an override.
Repository Layout
checkpoint/
model-00001-of-00003.safetensors
model-00002-of-00003.safetensors
model-00003-of-00003.safetensors
model.safetensors.index.json
negative_prompt.safetensors
release-manifest.json
config.json
Intended Use
TalkingAvatar is intended for research, evaluation, and authorized creative work with audio-driven avatars. Use only images and voices for which you have the necessary consent and rights, and disclose generated media where appropriate.
Limitations
Output quality and lip synchronization vary with pose, occlusion, audio quality, language, and image domain. The runtime does not provide identity verification, consent management, watermarking, or content moderation.
License
TalkingAvatar is released under the Apache License 2.0. See the source repository for complete license terms, third-party notices, and the full model card.
Acknowledgments
TalkingAvatar was developed as part of the 첨단 GPU 활용 지원 사업 (project period 2026-04-01 – 2026-07-15), supported by 정보통신산업진흥원 and 한국정보통신진흥협회.