pi05-isaaclab-ur7e-water-pour
A π₀.₅ (PI05) vision-language-action policy fine-tuned with LeRobot on a scripted IsaacLab watering task for a UR7e arm with an RH-P12-RN (rh5dg2) gripper.
Task prompt: Pick up the watering can and water the plant in the pot.
Model details
| Base model | lerobot/pi05_base |
| Policy type | pi05 |
| VLM backbone | PaliGemma gemma_2b |
| Action expert | gemma_300m |
| Training mode | train_expert_only=true (VLM backbone frozen, action expert trained) |
| Precision | bfloat16 |
| Action chunk | 50 steps (chunk_size=50, n_action_steps=50) |
| Flow-matching inference steps | 10 |
| State / action dim | 19 / 19 (padded to max_state_dim=32) |
| Cameras | third_person, eye_in_hand — 480×640 → resized to 224×224 |
| Normalization | VISUAL: IDENTITY, STATE: QUANTILES, ACTION: QUANTILES |
Observation / action spaces
Inputs:
observation.images.third_person—(3, 480, 640)observation.images.eye_in_hand—(3, 480, 640)observation.state—(19,)
Output:
action—(19,), emitted as a chunk of 50 future steps
Training data
ICRA2027-CSI/isaaclab-ur7e-water-pour-scripted
— scripted demonstrations collected in IsaacLab.
| Robot type | ur7e_rh5dg2 |
| Episodes | 100 |
| Frames | 36,192 |
| FPS | 50 |
| Tasks | 1 |
| LeRobot dataset version | v3.0 |
Training procedure
| Hyperparameter | Value |
|---|---|
| Steps | 15,000 |
| Batch size | 16 |
| Epochs | ≈ 6.63 (240,000 samples seen) |
| Optimizer | AdamW, β = (0.9, 0.95), ε = 1e-8, weight decay 0.01 |
| LR | 2.5e-5 with 1,000-step warmup, cosine decay over 30,000 steps to 2.5e-6 |
| Grad clip | 1.0 |
| Gradient checkpointing | enabled |
torch.compile | enabled (max-autotune) |
| Seed | 1000 |
Final training metrics at step 15,000:
| Metric | Value |
|---|---|
train/loss | 0.0094 |
train/grad_norm | 0.217 |
Checkpoints were saved every 5,000 steps; this repo contains the final step-15,000 checkpoint.
Training command:
bash scripts/train_pi05_isaaclab.sh \
--dataset-repo-id ICRA2027-CSI/isaaclab-ur7e-water-pour-scripted \
--dataset-root /path/to/lerobot/isaaclab-ur7e-water-pour-scripted \
--output-dir outputs/pi05-isaaclab-ur7e-water-pour \
--job-name pi05_isaaclab_ur7e_water-pour
Usage
from lerobot.policies.pi05.modeling_pi05 import PI05Policy
from lerobot.processor import make_pre_post_processors
policy = PI05Policy.from_pretrained("ICRA2027-CSI/pi05-isaaclab-ur7e-water-pour")
preprocessor, postprocessor = make_pre_post_processors(
policy.config,
pretrained_path="ICRA2027-CSI/pi05-isaaclab-ur7e-water-pour",
)
batch = {
"observation.images.third_person": third_person_image, # (B, 3, 480, 640)
"observation.images.eye_in_hand": wrist_image, # (B, 3, 480, 640)
"observation.state": state, # (B, 19)
"task": "Pick up the watering can and water the plant in the pot.",
}
action = postprocessor(policy.select_action(preprocessor(batch)))
The preprocessor tokenizes the task string with the google/paligemma-3b-pt-224 tokenizer
(max_length=200, right padding), so the language prompt must be supplied under the task key.
Limitations
- Trained only on scripted IsaacLab demonstrations for a single task — no real-robot data, no teleoperated data, and no task diversity. Expect a substantial sim-to-real gap.
- Only the action expert was trained; the frozen VLM backbone limits how much the policy can generalize to prompts outside the single training instruction.
- Trained for ~6.6 epochs over 100 episodes; no held-out validation or success-rate evaluation is reported here.
Files
| File | Description |
|---|---|
model.safetensors | Policy weights (~7.5 GB, bf16) |
config.json | PI05 policy config |
train_config.json | Full LeRobot training config used for this run |
policy_preprocessor.json + policy_preprocessor_step_2_normalizer_processor.safetensors | Input pipeline & normalization stats |
policy_postprocessor.json + policy_postprocessor_step_0_unnormalizer_processor.safetensors | Action un-normalization |