Qwen3-1.7B · ScienceWorld · trajectory SFT on gpt-5.4-mini rollouts — separate 1-epoch run

Student: Qwen/Qwen3-1.7B. Part of a study comparing teacher-trajectory SFT, online multi-turn ROSE and on-policy distillation for multi-turn agents.

Training

Trajectory SFT: one row per full teacher episode in ChatML, cross-entropy only on each teacher reply and its <|im_end|> (observations, instruction and assistant header are context). Batch 32, AdamW lr 1e-5, weight decay 0, cosine schedule with 10% warmup, fp32 training. Nothing filtered: failed episodes are included. Data: 2059 training task variations played once by gpt-5.4-mini (OpenAI API, reasoning_effort=medium; only the visible reply is used), teacher success 41.96%, 46035 supervised turns. 1 epoch = 64 steps with its own full cosine decay (unlike …-sft-gpt54mini-ep1, which is a mid-schedule checkpoint of the 3-epoch run). Stored in bf16.

Evaluation

ScienceWorld test split (AgentGym, 200 task variations), avg of 4 passes, temperature 0.4, 512 tokens/turn, 30 rounds, thinking off, no system turn. Success 9.12% ± 1.78, Avg@1 0.1467. Reference points on the same protocol: base Qwen3-1.7B 0.12%, Qwen3-32B itself 15.62%.

Notes

The teacher copies the prompt's action template open/close OBJ literally in 29.89% of its turns (always rejected by the simulator); this model emits it on 66.64% of its turns.

Prompt format

ReAct, multi-turn chat: the environment's AgentGym instruction as the first user turn, a canned assistant acknowledgement, then one user turn per observation; the model replies Thought:\n...\n\nAction:\n<one action>. Render with Qwen3's chat template and enable_thinking=False.