Qwen3-1.7B · ALFWorld + ScienceWorld + SearchQA · mixed trajectory SFT on Qwen3-32B rollouts — 1 epoch

Student: Qwen/Qwen3-1.7B. Part of a study comparing teacher-trajectory SFT, online multi-turn ROSE and on-policy distillation for multi-turn agents.

Training

Trajectory SFT: one row per full teacher episode in ChatML, cross-entropy only on each teacher reply and its <|im_end|> (observations, instruction and assistant header are context). Batch 32, AdamW lr 1e-5, weight decay 0, cosine schedule with 10% warmup, fp32 training. Nothing filtered: failed episodes are included. Data: the three environments' Qwen3-32B teacher trajectories mixed into one set (10479 episodes), 1 epoch = 327 steps, fp32.

Evaluation

Same per-environment protocols as the single-environment models (avg of 4 passes, T 0.4, thinking off): ALFWorld 48.50%, ScienceWorld 14.37%; the SearchQA evaluation was paused after 165 episodes and is not reported.

Prompt format

ReAct, multi-turn chat: the environment's AgentGym instruction as the first user turn, a canned assistant acknowledgement, then one user turn per observation; the model replies Thought:\n...\n\nAction:\n<one action>. Render with Qwen3's chat template and enable_thinking=False.