τ₀-World Model
This repository contains the official checkpoints for τ₀-World Model: A Unified Video-Action World Model for Robotic Manipulation.
τ₀-World Model (τ₀-WM) is a unified video-action world model that integrates policy learning, video prediction, and action evaluation within a single future-predictive framework. Built on a shared video diffusion backbone (Wan-2.2), τ₀-WM provides two complementary interfaces:
- Video Action Model (VAM): Jointly predicts future visual latents and continuous action chunks from multi-view observations, language instructions, and robot state.
- Action-conditioned Video Simulator: Rolls out candidate action chunks into multi-view futures and predicts dense task-progress scores.
The model is trained on approximately 27,300 hours of diverse data including real-robot teleoperation, UMI-style interaction, and egocentric human videos.
Released Checkpoints
This repository contains two complementary checkpoints of τ₀-WM:
| Component | Repository Path | Description |
|---|---|---|
| Video Action Model (VAM) | Repository root | Jointly predicts future visual latents and continuous action chunks from multi-view observations, language instructions, and robot states. |
| Action-conditioned Video Simulator | simulator/ | Rolls out candidate action chunks into multi-view futures and predicts dense task-progress scores. |
Download Checkpoints
Video Action Model
The Video Action Model is stored in the repository root. Download it without the simulator checkpoint using:
hf download sii-research/tau-0-wm \
--exclude "simulator/**" \
--local-dir checkpoints/tau-0/vam
Action-conditioned Video Simulator
The simulator is stored under the simulator/ directory. Download only the simulator checkpoint using:
hf download sii-research/tau-0-wm \
--include "simulator/**" \
--local-dir checkpoints/tau-0
Action Space
The model is designed for dual-arm manipulation with the following specifications:
- State Input: Absolute poses of two end-effectors (14 channels: xyz + xyzw quaternion) and gripper state (2 channels).
- Action Output: Absolute poses of end-effectors and gripper openness (16 channels total).
- Internal Optimization: During pretraining, the model is optimized to predict relative poses (20 channels: xyz and 6d-rotation for each arm).
For detailed deployment instructions and environment setup, please refer to the official GitHub repository.
Citation
@article{zhou2026tau0wm,
title={$\tau_0$-WM: A Unified Video-Action World Model for Robotic Manipulation},
author={Zhou, Pengfei and Shengcong Chen and Di Chen and Jiaxu Wang and Rongjun Jin and Bingwen Zhu and Yike Pan and Songen Gu and Kuanning Wang and Shufeng Nan and Xingyu Qiu and Chenhao Qiu and Pu Yang and Yunuo Cai and Jianxiong Gao and Yifan Li and Yanwei Fu and Xiangyu Yue and Zhi Chen and Jianlan Luo},
journal={arXiv preprint arXiv:2606.01027},
year={2026}
}