T-Rex + tactile — XHand1 on DexMate Vega

A DexMate Vega-1 with two RobotEra XHand1 hands holds a backpack open with the left hand and puts a mug inside it with the right (bimanual). Meta-glove teleop (no exoskeleton), Vive wrist tracking.

• Model T-Rex (Qwen3-VL-2B + mixture-of-transformer experts, cascaded flow matching, FLARE), fine-tuned from the released midtrain checkpoint. Tactile: 30-D fingertip force (10 fingers x 3) as a vector plus per-finger VQ-VAE codes (K=64, window 16).
• Action 42-D per step = 2 x (9-D end-effector pose in the chunk-start frame + 12 absolute hand-joint targets), chunk 16. Cameras: head-left, both wrists (384x288).
• Data 31 episodes; 27 train / 4 held out (every 10th).
• Training 10,000 steps, seed 1000, LR 1e-4. 4 GPUs x batch 16 (global 64).

Held-out open-loop error (mean ± SEM over held-out episodes)

Hands: mean |pred − commanded| over the 12 hand joints (rad). Arms: end-effector position (cm) and rotation (deg) error. The model sees the real observation every 16 steps; all 16 predicted steps are kept.

This measures trajectory tracking, not task success; nothing here was run on hardware.

Loading needs the XHand adaptation of the T-Rex code (42-D action, tacf6_dim=3); the stock release expects the 62-D Sharpa layout and will not load these weights. Files: model.pt, processor/, training_args.json, stats_data.json.

Related runs

• ckpt_mugbackpackteleop_place_4cam260918_trextactile260924
• ckpt_mugbackpackteleop_place_4cam260918_gr00t3b260924
• ckpt_mugbackpackteleop_place_4cam260918_gr00t3btactile260924
• ckpt_mugbackpackteleop_place_4cam260918_pi05260924
• ckpt_mugbackpackteleop_place_4cam260918_pi05tactile260924
• ckpt_mugbackpackteleop_place_4cam260918_act260924
• ckpt_mugbackpackteleop_place_4cam260918_acttactile260924
• ckpt_mugbackpackteleop_place_4cam260918_dp260925
• ckpt_mugbackpackteleop_place_4cam260918_dptactile260925