Latent-ECoT (Coconut) — MiniVLA-1B on LIBERO-90
Latent embodied chain-of-thought VLA policy: instead of generating ~250 tokens of text reasoning per decision (explicit ECoT, ~3 s), this policy reasons through 14 continuous latent "thought" vectors (Coconut-style hidden-state feedback) and decodes actions directly (~0.32 s/decision).
- Base: MiniVLA-1B (Qwen2.5-0.5B + DINOv2/SigLIP 224px), VQ action chunking (10 steps -> 7 codes)
- Trained 140k steps (batch 128) on LIBERO-90 with the Embodied-CoT reasoning annotations, via an 8-stage latentization curriculum (full-text ECoT -> all-latent)
- LIBERO-90: 89.1% (802/900), vs. explicit ECoT 85.2--87.0% and the ECoT-Lite paper's no-CoT Standard-VLA baseline of 82.0% (arXiv:2505.08243, Table 1)
Usage
Requires the latent-ecot codebase. Place the vq/
folder from this repo at the working directory root (the action tokenizer resolves it relatively),
then:
from prismatic.models import load_vla
vla = load_vla("<repo_dir>/checkpoints/step-140000-epoch-31-loss=0.4093.pt",
hf_token="", load_for_training=False) # auto-detects Coconut from config.json
action_chunk, info = vla.predict_action(
image, instruction, unnorm_key="libero_lm_90",
use_latent_cot=True, return_action_chunk=True, return_info=True)
Checkpoint is model-only (optimizer state stripped). See the GitHub repo for training/eval scripts, per-task results, and the supplementary write-up.
Citation
Please cite Coconut (arXiv:2412.06769), ECoT-Lite (arXiv:2505.08243), ECoT (arXiv:2407.08693), MiniVLA, and LIBERO alongside this artifact.