HIL-SERL (SAC + RLPD) — Franka Panda PickCube
A vision-based manipulation policy trained with HIL-SERL (SAC + RLPD) in the LeRobot
gym-hil simulation (PandaPickCube), fully autonomously on a single GPU — the MuJoCo
environment supplies the reward, so no human intervention or reward classifier is needed in sim.
Built for Session 3 (Robotics) of the Vizuara RL in Production workshop — the same stack trains a real SO-101 arm from scratch.
- Observation: front + wrist cameras (128×128) + 18-D proprioceptive state
- Action: 3-D end-effector delta + discrete gripper
- Algorithm: SAC, RLPD 50/50 online/offline mixing, LayerNorm critics
- Demos (offline seed):
lilkm/pick_cube_franka_panda_30
Load & evaluate
import torch, draccus
from huggingface_hub import snapshot_download
from lerobot.rl.train_rl import TrainRLServerPipelineConfig
from lerobot.rl import gym_manipulator as gm
from lerobot.processor import TransitionKey
from lerobot.policies.gaussian_actor.modeling_gaussian_actor import GaussianActorPolicy
local = snapshot_download("dizzlerama/hilserl-panda-pickcube-sac")
cfg = draccus.parse(TrainRLServerPipelineConfig, args=["--config_path", f"{local}/train_config.json"])
cfg.env.task = "PandaPickCube-v0"
policy = GaussianActorPolicy.from_pretrained(local).to("cuda").eval()
env, teleop = gm.make_robot_env(cfg.env)
env_p, act_p = gm.make_processors(env, teleop, cfg.env, "cuda")
# roll out with policy.select_action(...) — see the workshop notebook.
Reading: HIL-SERL (arXiv:2410.21845) · RLPD (arXiv:2302.02948) · LeRobot HIL-SERL docs