OpenVLA-7B — ALOHA coffee (full fine-tune)
Full-parameter fine-tune for bimanual coffee-making. Self-contained weights.
Fine-tuned from openvla/openvla-7b on a curated
subset of lerobot/aloha_static_coffee
— a bimanual ALOHA arm making coffee.
Read this before loading the weights
action_stats.json in this repo is required to decode the model's output. The action head emits
bin indices, not joint values. Turning a bin back into a joint target needs the exact q01/q99
statistics the model was trained against, which are in that file. Weights without it are not a
usable policy.
from transformers import AutoModelForVision2Seq, AutoProcessor
import json
REPO = "anjali209/openvla-7b-aloha-coffee-full"
processor = AutoProcessor.from_pretrained("openvla/openvla-7b", trust_remote_code=True)
model = AutoModelForVision2Seq.from_pretrained(REPO, dtype="bfloat16", trust_remote_code=True)
# required to decode bins -> joint targets
from huggingface_hub import hf_hub_download
stats = json.load(open(hf_hub_download(REPO, "action_stats.json")))
Training
| Base model | openvla/openvla-7b (7.54B params) |
| Trainable | 7,541,237,184 (100%) |
| Data | 42,006 train frames (48 episodes) / 1,831 val (2 held-out episodes) |
| Camera | cam_left_wrist — a single wrist (egocentric) view |
| Steps | 5,250 = 1.00 epoch |
| Precision | bf16, gradient checkpointing on |
| Optimizer | AdamW, OneCycleLR (pct_start=0.03), grad clip 1.0 |
| Learning rate | 2e-5 |
| Peak GPU memory | 38.0 GB (2x H100, device_map sharded) |
| Wall time | 72.7 min |
| Checkpoint selected on | val_action_mae (best 0.0531 @ step 4500) |
Actions are 14-D joint targets, normalised per dimension with q01/q99 to [-1, 1], discretised into 256 uniform bins, and mapped onto the least-used token ids in the vocabulary (31744-31999). Loss is cross-entropy over the 14 action tokens with the prompt masked.
Splits are by episode, not by frame — 65-75% of wrist frames are visually near-static, so a frame-level split would put near-duplicates on both sides.
Limitations
Please read these before using the model for anything.
- Not validated on a robot or in simulation. All evaluation was offline and teacher-forced: the model saw a real observation every frame and its predictions drove nothing, so compounding error is not measured at all.
- It does not complete the task on held-out episodes. Scored on a milestone criterion (pod grasped -> placed -> basket grasped -> seated, in order), it reaches all four milestones but fails the ordering/hold check on both held-out episodes: the predicted gripper closes on the basket and reopens 1-2 frames later.
- The gripper command is temporally inconsistent. Each action is an independent per-frame argmax over 256 bins, with nothing tying consecutive frames together. The predicted gripper moves up to 0.48 in a single frame against a demonstration maximum of 0.036. On hardware that is a command to snap the gripper open and shut at 50 Hz.
- Large train/held-out gap. The same criterion scores 85-92% on the 48 training episodes and 0/2 on the 2 held-out ones — the model fits the demonstrations it saw and does not transfer.
- One camera of four. Only
cam_left_wristis used. Notably, the left arm handles the first sub-task while the right arm performs the one the model fails on. - Action MAE is not a good proxy for competence here. A "repeat the previous frame's action" baseline beats this model 26x on that metric.
The practical next step this points at is constraining the gripper channel temporally at decode time (action chunking, hysteresis, or a loss term penalising frame-to-frame gripper jumps).
Provenance
Trained as part of a data-pipeline exercise covering dataset audit, labeling design, curation, evaluation design, and VLA adaptation. Training data came from a curated build: post-completion motion trimmed (~20% of frames) and a measured 5-frame video/state offset corrected before training.