Model Card for VQ-BeT/PushT
VQ-BeT (as per Behavior Generation with Latent Actions) trained for the PushT environment from gym-pusht.
How to Get Started with the Model
See the LeRobot library (particularly the evaluation script) for instructions on how to load and evaluate this model.
Training Details
Trained with LeRobot@342f429.
The model was trained using this command:
python lerobot/scripts/train.py \
policy=vqbet \
env=pusht dataset_repo_id=lerobot/pusht \
wandb.enable=true \
device=cuda
The training curves may be found at https://wandb.ai/jaylee0301/lerobot/runs/9r0ndphr?nw=nwuserjaylee0301.
Training VQ-BeT on PushT took about 7-8 hours to train on an Nvida A6000.
Model Size
| Number of Parameters | |
|---|---|
| RGB Encoder | 11.2M |
| Remaining VQ-BeT Parts | 26.3M |
Evaluation
The model was evaluated on the PushT environment from gym-pusht. There are two evaluation metrics on a per-episode basis:
- Maximum overlap with target (seen as
eval/avg_max_rewardin the charts above). This ranges in [0, 1]. - Success: whether or not the maximum overlap is at least 95%.
Here are the metrics for 500 episodes worth of evaluation.
| Ours | |
|---|---|
| Average max. overlap ratio for 500 episodes | 0.895 |
| Success rate for 500 episodes (%) | 63.8 |
The results of each of the individual rollouts may be found in eval_info.json.