VLAP4P FR3 policy — Azure-camera variant (run 20261003T230324Z, step 310000)
OpenVLA-OFT fine-tuned for nine tabletop pick-and-place tasks on a Franka FR3, for the real cell's Azure Kinect overhead camera. University of Auckland Part IV project #40 (2026), code: the VLAP4P repository.
Two different policies share the step number 310000. This is the Azure-camera policy:
fr3_azure_overhead/20261003T230324Z--310000, camera rigfr3_azure_overhead_v1. The reference-camera policy (fr3_tabletop_multitask/20260919T193159Z--310000, rigfr3_reference_vision, repotaalyelxor/vlap4p-fr3-310000) produced every simulation result in the project report and was trained for a different, simulator-designed camera pose. Do not swap them, and always name a policy by its run and camera rig, not by its step.
What changed from the reference policy
Only the overhead camera. The same 125 human demonstrations and 496 accepted Isaac Lab Mimic
demonstrations (621 trajectories, 115,604 steps) were re-rendered from saved simulator
states with an overhead camera placed to match two photographs of the real cell
(configs/azure_overhead/README.md). Actions, proprioception, instructions and the wrist view are
unchanged. Training recipe unchanged: LoRA rank 32, batch 8, learning rate 5e-5, image
augmentation, 10,000 FR3 steps on top of the upstream LIBERO checkpoint's 300,000 (hence "310000").
The camera pose is photo-matched, not a measured calibration. Before using this policy on a
robot, run the camera check in the FR3 console (scripts/fr3_console.py, docs/FR3_REAL_BRINGUP.md).
Required image preprocessing (the policy fails silently without it)
The overhead image the policy was trained on is not the raw camera frame:
- raw Azure Kinect colour frame, 16:9 (trained at 1280×720), RGB;
- centre square crop (720×720 at 1280×720);
- rotate 180°;
- resize to 256×256 (bilinear);
- then OpenVLA-OFT's own 90% centre crop and resize to 224×224.
policy_artifact.json records this as preprocessing.overhead_rotate_crop_256: true; the
repository's FR3 runtime applies steps 2–5 from the artifact
(policy_core/camera_preprocessing.py) and refuses an artifact whose preprocessing it does not
implement. The wrist image (640×480) is used as recorded.
Files
| Path | What |
|---|---|
data/experiments/fr3_azure_overhead/20261003T230324Z--310000_chkpt/ | merged weights, LoRA adapter, action head, proprio projector, policy_artifact.json |
data/experiments/fr3_azure_overhead/20261003T230324Z/ | run manifest (exact training command), completion manifest (checkpoint SHA256s), dataset statistics |
configs/azure_overhead/fr3_azure_overhead_vision.yaml | the camera rig the policy was trained for |
data/monitors/isaac_fr3_azure_* | supervisor files calibrated for this policy (added after the overnight 4–5 Oct runs) |
tests/fixtures/fr3_policy_golden_azure.* | fixed inputs and this policy's outputs, for the golden test on another computer |
BUNDLE.json | git commit of the uploading checkout |
Checkpoint SHA256 (completion manifest): 42f1729d4c01d5abbac2744c09c05655a0a99376ebfb2da2c7163aba2fe69d4a.
Camera rig hash: 02a139b948d6a93b8ccd7390464cf0724bd5d82ad9380b25fb07b1d37ee8d648.
Evaluation so far
Simulation only, with the photo-matched camera. Quick check on bowl on plate (16 episodes,
seeds 0–15, strict success): checkpoints 302500 / 305000 / 307500 / 310000 succeeded 8 / 13 /
11 / 13 times. A full nine-task evaluation and supervised arms ran overnight 4–5 Oct; results go
in docs/AZURE_CAMERA_POLICY.md. The real-robot evaluation is pending.
Use
git clone <VLAP4P repository> && cd VLAP4P
scripts/bootstrap.sh --role fr3 # environment, this policy, smoke tests
.venv/bin/python scripts/fr3_console.py --artifact data/experiments/fr3_azure_overhead/20261003T230324Z--310000_chkpt/policy_artifact.json
Requires transformers 4.40.1 (other versions run but change the actions) and a GPU with about 16 GB free.
License and provenance
Derived from OpenVLA-OFT (MIT), whose language backbone is Llama 2: use is subject to the Llama 2 Community License. Training data are simulated (Isaac Lab); no real-world images or people are included.
Compendium access and policy identity
Public research download; no account, token or access approval is required. The weights inherit the Llama 2 Community License; see LICENSE and NOTICE. Acceptable use: https://ai.meta.com/llama/use-policy
The hardware experiment used the real-bowl policy fr3_real_bowl_oft/20261005T130437Z-real-bowl--310000 in taalyelxor/real-fine-tuned, initialised from measured-camera step 305000 and trained on eight real demonstrations (two of ten recordings held out). Azure and measured-camera comparison results are simulation evidence. Any earlier card text naming the Azure policy as the hardware policy is superseded.