VLADrop-OpenVLA-OFT-LIBERO-mha-only
Checkpoint for Drop-Then-Recovery: How Redundant Are Vision-Language-Action Models?.
DTR (Drop-Then-Recovery) removes transformer blocks from a pretrained VLA model and recovery-fine-tunes the smaller dense model. Code: https://github.com/s1ghhh/VLADrop
This checkpoint
| Paper row | Table 2: OpenVLA-OFT MHA (granularity study) |
| Dropped blocks | Llama-2 backbone (32 layers): drop only the MHA sublayers of the 16 odd-indexed blocks [1,3,5,...,31]. MLP sublayers kept. |
| Recovery training | LoRA rank 32 (merged), batch size 16, 50K steps, lr 5e-4 |
| LIBERO success rate | Spatial 95.0 / Object 99.2 / Goal 99.2 / Long 76.4 / Avg 91.9 |
Usage
Merged full-weight OpenVLA-OFT checkpoint. The drop lists are baked into
config.json (text_config.drop_attn_list / drop_mlp_list), so evaluation
reconstructs the pruned graph automatically. Requires the VLADrop modified
transformers (transformers-openvla-oft-dropped) from https://github.com/s1ghhh/VLADrop
python -m experiments.robot.libero.run_libero_eval_random_dropped \
--pretrained_checkpoint <this_repo_local_path> \
--task_suite_name libero_spatial \
--num_trials_per_task 50 --center_crop True
Includes action_head--*.pt and proprio_projector--*.pt (L1 regression head, 2 input images + proprio).
Citation
@article{sun2026vladrop,
title={Drop-Then-Recovery: How Redundant Are Vision-Language-Action Models?},
author={Sun, Guoheng and Feng, Kaixi and He, Shwai and Gong, Xiaochuan and He, Yexiao and Wang, Ziyao and Shen, Zheyu and Ye, Wanghao and Kompella, Ramana Rao and Liu, Gaowen and Li, Ang},
journal={arXiv preprint arXiv:2606.27755},
year={2026}
}