cosmos-predict2.5-2b-agibot-mv-robocasa-pressbutton

Private research checkpoints — Cosmos-Predict2.5-2B robot/multiview-agibot (frameinit) post-trained on RoboCasa CoffeePressButton 3-view renders (55 human demos, 704² → trained at 480², 81 frames/view @20 fps). 2026-10.

• Base: nvidia/Cosmos-Predict2.5-2B robot/multiview-agibot/f740321e-…_ema_bf16.pt (full fine-tuning, no LoRA, EMA rate 0.1)
• Views: head←robot0_agentview_left, hand_0←robot0_eye_in_hand (wrist), hand_1←robot0_agentview_right
• Camera: intrinsics from MuJoCo fovy (left/right 60°, wrist 75°), extrinsics = identity (w2c)
• Text: Reason1 embeddings precomputed, caption "press the button on the coffee machine to serve coffee"
• Optim: AdamW lr 4.3e-5, wd 1e-3, warmup 200, cosine (f_max .5 / f_min .2) over 20,000 iters, global batch 2 (2×A100 40GB, 28.7 s/iter)
• Data: Gom-sy/robocasa-coffee-v2w-704 (press_button subset)

Files

iter_XXXXXX/model_ema_bf16.pt — EMA weights (bf16) for inference with experiment 2b_robocasa_coffee_agibot_frameinit (code/ holds the dataset/experiment files and mv_infer.py needed to reproduce; see code/HANDOFF.md).

Intended use

Research only (world-model video generation for IDM labelling / policy training).

Round 2 (realcam/) — real camera poses (2026-10-05 →)

Round 1 (iter_004000/, identity extrinsics) showed view collapse at t≥20 frames (3 views converging to one), seed-dependent. Round 2 restarts from the agibot base with real MuJoCo camera poses: per-frame world-to-camera extracted from the rendered hdf5 states (cameras/{tag}_{idx}.npz), expressed relative to the head camera at the conditioning frame (head = I, right = fixed rig offset 0.7 m, wrist = arm pose), repeated for the L latent frames; OpenGL→OpenCV flip diag(1,-1,-1). Everything else identical (81 f / 480², press_button 55 ep, 20k iters, ckpt every 4k). Files: realcam/iter_XXXXXX/model_ema_bf16.pt. Inference: code/mv_infer.py --cam real (views are stacked along HEIGHT in the decoded output).