Qwen3-1.7B · ALFWorld · trajectory SFT on Qwen3-32B rollouts — epoch 1 of 3
Student: Qwen/Qwen3-1.7B. Part of a study comparing teacher-trajectory SFT, online multi-turn ROSE and on-policy distillation for
multi-turn agents.
Training
Trajectory SFT: one row per full teacher episode in ChatML, cross-entropy only on each teacher reply and its <|im_end|> (observations, instruction and assistant header are context). Batch 32, AdamW lr 1e-5, weight decay 0, cosine schedule with 10% warmup, fp32 training. Nothing filtered: failed episodes are included. Data: 2420 training games played by Qwen3-32B (teacher success 55.70%). One 3-epoch run (225 steps, cosine over all 225); this is step 75. Stored in bf16.
Evaluation
ALFWorld test split (AgentGym, 200 games), avg of 4 passes, temperature 0.4, 512 tokens/turn, 30 rounds, thinking off. Success: not evaluated. Epoch 3 of this run scores 52.88% (vs the separate 1-epoch run 48.75%: +4.12, 95% CI [0.88, 7.38]). Reference points on the same protocol: base Qwen3-1.7B 19.38%, Qwen3-32B itself 52.12%.
Notes
Epochs 1–2 are mid-schedule checkpoints and were not evaluated.
Prompt format
ReAct, multi-turn chat: the environment's AgentGym instruction as the first user turn, a canned assistant acknowledgement, then one user
turn per observation; the model replies Thought:\n...\n\nAction:\n<one action>. Render with Qwen3's chat template and enable_thinking=False.