Anchor-Align: Generalizable VLA Finetuning via Representation Anchoring and Language-Action Alignment

🌐 Project Page  •  📄 Paper (arXiv)  •  💻 Code (GitHub)  •  🎬 Videos

TL;DR — Standard behavior-cloning finetuning of a vision-language model on robot demos silently erases the pretrained VLM's semantics and decouples its language output from its actions. Anchor-Align adds two lightweight losses on top of action prediction that (i) anchor the trainable VLA to a frozen copy of the pretrained VLM (preserving vision-language reasoning) and (ii) align the pre-action hidden state with a discrete motion-direction label derived from the executed motion. The result: 22.6% on LIBERO-PRO position swap (vs 2.3% for standard BC), 90.3% LIBERO-Plus average (vs 85.1%), and near-doubled real-world success (28.3% → 54.2%) on a UFactory xArm7 setup.

📦 Companion GitHub Repository

This repo hosts the weights. The inference and evaluation code lives at github.com/dwipddalal/Anchor-Align.

Note: the training code is not part of the initial release; we plan to release it in a follow-up update. Each checkpoint's full training configuration is documented in its MODEL_CARD.md.

Available Checkpoints

All four finetune Prismatic Qwen2.5-0.5B (DINOv2 + SigLIP, 24 layers) with LoRA rank 64 and an L1-regression MLPResNet action head. Per-checkpoint metrics live in each subfolder's MODEL_CARD.md; reproduction targets and tolerances are in the GitHub repo's REPRODUCE.md.

Results

Tables below are reproduced from the paper. Reproduction targets and tolerances for the released checkpoints are in REPRODUCE.md.

Standard LIBERO suites

Success rates on the four standard (unperturbed) LIBERO suites. Anchor-Align achieves the highest success rate on every suite, surpassing methods with substantially larger backbones and large-scale robot-action pretraining.

Robustness and generalization — LIBERO-PRO and LIBERO-Plus

Success rates under perturbation on the LIBERO-Spatial suite (paper Table 1). Bold = best, underline = second best.

*Our implementation of knowledge insulation adapted to VLA-Adapter. Position swap is the hardest axis: MolmoAct and OpenVLA-OFT score 0%, standard BC reaches 2.3%, while Anchor-Align reaches 22.6%.

Qualitative — generalization to semantic perturbations

Same task, different phrasing / different object identity / shuffled positions. Anchor-Align retains the VLM's semantic understanding of what the instruction refers to, while the baseline latches onto memorized appearance shortcuts.

Per-suite robustness — Object, Goal, and Long

The same gains carry over to the remaining three LIBERO suites. Each radar plot compares Anchor-Align (orange) against the standard BC VLA-Adapter baseline (gray) across nine evaluation axes: two from LIBERO-PRO (Language Rephrase, Object Swap) and seven from LIBERO-Plus.

Largest gains: Robot Init State +18.6 on Object; Language Instruction +11.9 on Goal; Lighting Condition +20.8, Object Layout +18.6, and Camera Viewpoint +17.7 on Long.

Long-horizon generalization — CALVIN ABC→D

Each rollout chains five language instructions; k/5 is the fraction of rollouts completing the first k, and Len is the average number of consecutively completed tasks.

Multi-seed significance

Mean ± standard deviation over 5 training seeds on LIBERO-Spatial; the method gaps are far larger than seed-to-seed variability.

Per-folder layout

Each subfolder is a self-contained inference bundle:

<benchmark>_<steps>/
  MODEL_CARD.md
  model.safetensors                        # merged base VLM weights (~2.5 GB)
  lora_adapter/adapter_model.safetensors   # separate LoRA weights (~208 MB)
  action_head--<step>_checkpoint.pt        # L1 regression head (~436 MB)
  align_dir_proj--<step>_checkpoint.pt     # alignment direction projector
  proprio_projector--<step>_checkpoint.pt  # proprioception projector
  config.json, tokenizer.*, processor_config.json, dataset_statistics.json

Quickstart — download + run inference

1. Clone the code repo (contains all inference and eval scripts):

git clone https://github.com/dwipddalal/Anchor-Align.git
cd Anchor-Align
pip install -e .   # + follow the installation section in the GitHub README

2. Download a checkpoint from this repo:

from huggingface_hub import snapshot_download

local_dir = snapshot_download(
    repo_id="Dwipz/Anchor-Align",
    allow_patterns=["config.json", "libero-spatial/*"],   # or libero-object/*, libero-goal/*, libero-long/*
    local_dir="./checkpoints",
)

3. Run inference using the eval scripts in the GitHub repo:

# LIBERO Standard
CUDA_VISIBLE_DEVICES=0 python experiments/robot/libero/run_libero_eval.py \
  --pretrained_checkpoint ./checkpoints/libero-spatial \
  --task_suite_name libero_spatial \
  --use_proprio True --num_images_in_input 2 --use_pro_version True

# LIBERO-PRO (paraphrased-language perturbation)
CUDA_VISIBLE_DEVICES=0 python experiments/robot/libero_pro/run_libero_pro_eval.py \
  --pretrained_checkpoint ./checkpoints/libero-spatial \
  --base_suite_name libero_spatial --perturbation_type lan \
  --use_proprio True --num_images_in_input 2 --use_pro_version True

Inference flags (--use_proprio True --num_images_in_input 2 --use_pro_version True --use_l1_regression True --center_crop True --num_open_loop_steps 8) match the paper. See the GitHub README's "Evaluation" section for LIBERO-Plus and batched variants.

License

MIT. See the GitHub repository LICENSE.

Citation

@article{dalal2026anchoralign,
  title   = {Generalizable VLA Finetuning via Representation Anchoring and Language-Action Alignment},
  author  = {Dalal, Dwip and Patel, Shivansh and Jain, Chahit and Kim, Jeonghwan and Mishra, Utkarsh and Baratian, Alex and Ha, Hyeonjeong and Ji, Heng and Lazebnik, Svetlana and Jain, Unnat},
  journal = {arXiv preprint arXiv:2607.13429},
  year    = {2026}
}

Keywords

Vision-language-action models, robot learning, robot manipulation, continual learning, catastrophic forgetting, out-of-distribution generalization, language-action alignment, behavior cloning, imitation learning, knowledge distillation, VLA finetuning.