GATS Cold-Start: Pure-OPD cold-start + GRPO (ALFWorld)
Part of the Cold-start collection for the GATS paper (capability-aware on-policy distillation). Student: Qwen2.5-7B-Instruct; teacher: Qwen2.5-1.5B GRPO-trained on ALFWorld train tasks.
Chain: pure-OPD cold-start -> pure GRPO
Starting point: the pure-OPD cold-start checkpoint (see
C12H18/gats-coldstart-opd-qwen2.5-7b-alfworld, N = 52 on-policy
distillation steps). Then pure GRPO:
- GRPO: 150 steps, lr 1e-6, group n=8, 16 tasks/step, gamma 0.95, KL loss (low-variance, coef 0.01), invalid-action penalty 0.1, entropy 0.001, grad clip 1.0
- Run id:
opdgrpo-alf7b-t1p5b-s0(seed 0) - Final evaluation: ID 86.20 ± 1.19 / OOD 88.80 ± 1.97 (K=3, rollout seeds 0/1/2)
Intended use
Research on cold-start strategies for agent RL (imitation-first vs RL-from-scratch) and reproduction of the GATS paper baselines. Not a general-purpose assistant.
Provenance
Trained 2026-09-28 on the internal verl-based stack; checkpoints uploaded unmodified from the training runs (bfloat16 unless noted float32 above).