VLADrop-pi05-LIBERO-drop9-language-odd
Checkpoint for Drop-Then-Recovery: How Redundant Are Vision-Language-Action Models?.
DTR (Drop-Then-Recovery) removes transformer blocks from a pretrained VLA model and recovery-fine-tunes the smaller dense model. Code: https://github.com/s1ghhh/VLADrop
This checkpoint
| Paper row | Table 1 'Drop Half Language' / Table 2 'Block' / Table 7 (9/18) |
| Dropped blocks | LLM (PaliGemma) blocks [1,3,5,7,9,11,13,15,17] (all odd-indexed; attn+mlp) |
| Recovery training | batch size 32, 30K steps, lr 5e-5 (10K warmup) |
| LIBERO success rate | Spatial 98.8 / Object 98.6 / Goal 93.4 / Long 82.4 / Avg 93.3 |
Usage
This is an openpi-format pi0.5 checkpoint (PyTorch). Use with the VLADrop fork: https://github.com/s1ghhh/VLADrop
python scripts/serve_policy_batch_drop.py \
--config pi05_libero_dropped \
--dir <this_repo_local_path> \
--port 8000
Important: the drop lists are NOT stored inside the checkpoint. Pass the exact
llm_drop_attn_list / llm_drop_mlp_list shown above (via config or CLI) when serving,
otherwise layers will be mismatched. assets/ contains the LIBERO norm stats.
The optimizer state (train_state/) is not included.
Citation
@article{sun2026vladrop,
title={Drop-Then-Recovery: How Redundant Are Vision-Language-Action Models?},
author={Sun, Guoheng and Feng, Kaixi and He, Shwai and Gong, Xiaochuan and He, Yexiao and Wang, Ziyao and Shen, Zheyu and Ye, Wanghao and Kompella, Ramana Rao and Liu, Gaowen and Li, Ang},
journal={arXiv preprint arXiv:2606.27755},
year={2026}
}