τ₀-VLA Checkpoint
This repository contains the official pretrained checkpoint for the low-level vision-language-action policy of τ₀-VLA: a Hierarchical Robot Foundation Model with World-Model-Guided Test-Time Computation. It is intended as an initialization for embodiment- and task-specific post-training.
The complete τ₀-VLA system additionally uses high-level proposal and reflection models, a world model, and a value model for test-time search. Those high-level components are described in the paper and are not part of this checkpoint.
Model releases
| Model | Use |
|---|---|
| τ₀-VLA | Pretrained low-level policy for robot post-training |
| τ₀-VLA LIBERO | 60,000-step LIBERO checkpoint with simulation evaluation |
Model summary
| Item | Description |
|---|---|
| Model type | Pretrained low-level vision-language-action policy |
| Vision-language backbone | Qwen3.5-2B |
| Action model | Mixture-of-Transformers action expert |
| Training objective | Conditional flow matching |
| Inputs | Multi-view RGB observations, language instruction or subtask, robot metadata, state, and masks |
| Output | A normalized 30 × 40 continuous action chunk |
| Inference integration | 10 flow-matching steps |
| Weight dtype | BF16 |
| Intended use | Robot-policy research and target-specific post-training |
The low-level policy was trained on 40,115 hours of heterogeneous real-world robot data with multimodal co-training. See the paper for the complete training procedure and system-level evaluation.
Post-training
The reference environment uses Python 3.11, CUDA 12.8, and PyTorch 2.7.1.
git clone https://github.com/sii-research/tau-0-vla.git
cd tau-0-vla
bash scripts/setup.sh
hf download sii-research/tau-0-vla \
--local-dir checkpoints/tau-0-vla-base
Post-train on the bundled AgiBot World example:
bash scripts/train.sh configs/example_agibot_world_gong/train.yaml \
--model_name_or_path checkpoints/tau-0-vla-base
The Python loader accepts either a Hub model ID or a local checkpoint path. The quick start downloads the checkpoint first because the training launcher uses offline Hub access by default. The example uses a small Strike the gong subset derived from AgiBot World Beta and stored in LeRobot v3.0 format.
For a new task or embodiment, start from:
Input and output contract
The model uses a unified 40-dimensional state/action layout:
| Slots | Semantic value | Representation |
|---|---|---|
0:3 | Left end-effector position | XYZ in metres |
3:9 | Left end-effector orientation | Rotation 6D |
9:12 | Right end-effector position | XYZ in metres |
12:18 | Right end-effector orientation | Rotation 6D |
18 | Left gripper | One scalar |
19 | Right gripper | One scalar |
20:22 | Waist | Up to two values |
22:24 | Chassis velocity | Up to two values |
24:32 | Left arm joints | Up to eight joints, in radians |
32:40 | Right arm joints | Up to eight joints, in radians |
Model-facing tensors have the following shapes:
state (40,) float32
action (30, 40) float32
state_mask (40,) float32
action_mask (40,) float32
State and action masks are independent. Inactive slots are zero and have mask
value 0; changing the mapping, mask, camera order, or normalization changes
the action semantics. Arm-joint and end-effector targets are relative to the
current state, while gripper, waist, and chassis-velocity targets remain
absolute. Adapters restore the output to native robot order after
unnormalization.
The public code includes the following stable unified mapping:
| Robot route | Registry key | Active control interface |
|---|---|---|
g1_agibot_unified | g1_agibot_36 | Dual 7-DoF arms and grippers |
Detailed contracts:
Scope and evaluation
This is a pretrained checkpoint intended to initialize task- and embodiment-specific fine-tuning. For downstream use, fine-tune it on data from the target task and deploy the resulting policy checkpoint. The closed-loop and test-time-computation results in the paper use task-specific fine-tuned policies and, where applicable, the complete hierarchical τ₀-VLA system.
Serving also requires the policy manifest, run specification, Data Spec, and normalization artifacts written by fine-tuning. Run the public server against the resulting fine-tuned policy output.
The bundled AgiBot World subset is a worked post-training example rather than the checkpoint's pretraining corpus.
Intended use and limitations
- Intended uses include robot-learning research, embodiment adaptation, and task-specific post-training.
- A target deployment must provide the correct adapter, camera ordering, 40D mapping, masks, normalization statistics, and native action restoration.
- Hardware serving uses joint-control checkpoints. The LIBERO checkpoint uses a dedicated EEF simulation server.
- The checkpoint is not demonstrated as a zero-shot controller for arbitrary robots or tasks.
- Before physical deployment, validate open-loop behavior and enforce workspace, velocity, collision, human-supervision, and emergency-stop safeguards.
Citation
@misc{cai2026tau0vla,
title = {{$\tau_0$-VLA: a Hierarchical Robot Foundation Model with
World-Model-Guided Test-Time Computation}},
author = {Xiaowei Cai and Yunuo Cai and Bingao Chen and Jingxiao Chen and
Zhi Chen and Siyuan Feng and Tengyu Hou and Jingshun Huang and
Han Jiang and Runkun Ju and Dong Li and Mingxiang Li and
Shaowei Li and Xinchen Li and Yifan Li and Yi Liu and
Zhongyuan Liu and Jianlan Luo and Junwen Miao and Ruiqi Ni and
Buqing Nie and Mingjie Pan and Xinlin Ren and Jianheng Song and
Jiaxu Wang and Peiqi Wang and Sen Wang and Xiaoyan Wang and
Dafeng Wei and Dongming Wu and Pengwei Xie and Pu Yang and
Hangjian Ye and Xiangyu Yue and Jinyu Zhang and Qinglin Zhang and
Xueyong Zhao and Yue Zhou},
year = {2026},
url = {https://arxiv.org/abs/2608.16885}
}
License
The model weights and source code are released under the Apache License 2.0. The bundled example data is separately licensed under CC BY-NC-SA 4.0.