GATS Cold-Start: SFT cold-start + GRPO (WebShop)
Part of the Cold-start collection for the GATS paper (capability-aware on-policy distillation). Student: Qwen2.5-7B-Instruct; teacher: Qwen2.5-1.5B GRPO-trained on WebShop train tasks.
Chain: SFT cold-start -> pure GRPO
Starting point: the SFT cold-start checkpoint (see
C12H18/gats-coldstart-sft-qwen2.5-7b-webshop). Then pure GRPO:
- GRPO: 150 steps, lr 1e-6, group n=8, 16 tasks/step, gamma 0.95, KL loss (low-variance, coef 0.01), invalid-action penalty 0.1, entropy 0.001, grad clip 1.0
- Run id:
sftgrpo-ws7b-t1p5b-s0(seed 0) - Final evaluation: 75.61 ± 3.44 (K=3, rollout seeds 0/1/2 × env seeds 0/1/2)
Cold-start phase recap
3 epochs, lr 1e-5, global batch 32 (micro-batch 8/GPU), FSDP, multiturn format with loss on assistant spans; corpus 6,129 rows / 240 tasks; checkpoint global_step_573.
Intended use
Research on cold-start strategies for agent RL (imitation-first vs RL-from-scratch) and reproduction of the GATS paper baselines. Not a general-purpose assistant.
Provenance
Trained 2026-09-28 on the internal verl-based stack; checkpoints uploaded unmodified from the training runs (bfloat16 unless noted float32 above).