GATS Cold-Start: SFT cold-start (ALFWorld)
Part of the Cold-start collection for the GATS paper (capability-aware on-policy distillation). Student: Qwen2.5-7B-Instruct; teacher: Qwen2.5-1.5B GRPO-trained on ALFWorld train tasks.
Cold-start phase: supervised fine-tuning
Fine-tuned on successful trajectories of a Qwen2.5-1.5B teacher (itself GRPO-trained for 150 steps on the ALFWorld train tasks). Corpus: teacher rollouts (18-21 sampling steps x 16 tasks x 8 rollouts per task), success-filtered, capped at 4 / floored at 2 trajectories per task, rendered as per-turn multiturn rows (loss on assistant spans); every turn is byte-verified against the teacher tokenizer's chat template.
- Corpus: 13,970 rows over 239 unique tasks
- Training: 3 epochs, lr 1e-5, global batch 32 (micro-batch 8/GPU), FSDP, multiturn format with loss on assistant spans
- Checkpoint: global_step_1308 ("128-task" corpus group)
- Validation before any RL: 64.1% (train-time val at RL step 0; ALFWorld out-of-distribution split)
Storage
Weights are stored in float32 (7 shards). Cast to bfloat16 for inference if preferred; this is the unmodified SFT trainer checkpoint.
Intended use
Research on cold-start strategies for agent RL (imitation-first vs RL-from-scratch) and reproduction of the GATS paper baselines. Not a general-purpose assistant.
Provenance
Trained 2026-09-28 on the internal verl-based stack; checkpoints uploaded unmodified from the training runs (bfloat16 unless noted float32 above).