GATS Cold-Start: SFT cold-start + GRPO (ALFWorld)
Part of the Cold-start collection for the GATS paper (capability-aware on-policy distillation). Student: Qwen2.5-7B-Instruct; teacher: Qwen2.5-1.5B GRPO-trained on ALFWorld train tasks.
Chain: SFT cold-start -> pure GRPO
Starting point: the SFT cold-start checkpoint (see
C12H18/gats-coldstart-sft-qwen2.5-7b-alfworld). Then pure GRPO:
- GRPO: 150 steps, lr 1e-6, group n=8, 16 tasks/step, gamma 0.95, KL loss (low-variance, coef 0.01), invalid-action penalty 0.1, entropy 0.001, grad clip 1.0
- Run id:
sftgrpo-alf7b-t1p5b-s0(seed 0) - Final evaluation: ID 77.86 ± 1.63 / OOD 83.85 ± 6.74 (K=3, rollout seeds 0/1/2)
Cold-start phase recap
3 epochs, lr 1e-5, global batch 32 (micro-batch 8/GPU), FSDP, multiturn format with loss on assistant spans; corpus 13,970 rows / 239 tasks; checkpoint global_step_1308.
Intended use
Research on cold-start strategies for agent RL (imitation-first vs RL-from-scratch) and reproduction of the GATS paper baselines. Not a general-purpose assistant.
Provenance
Trained 2026-09-28 on the internal verl-based stack; checkpoints uploaded unmodified from the training runs (bfloat16 unless noted float32 above).