Qwen3-1.7B · TextCraft · multi-turn on-policy distillation from Qwen3-32B — step 33 of 66
Student: Qwen/Qwen3-1.7B. Part of a study comparing teacher-trajectory SFT, online multi-turn ROSE and on-policy distillation for
multi-turn agents.
Training
Multi-turn OPD: the student plays whole episodes (30 turns) and is trained with a reverse-KL policy-gradient signal from Qwen3-32B on its own generated tokens only. lr 1e-6, weight decay 0.1, batch 32, 6 epochs over 374 tasks = 66 steps, bf16.
Evaluation
TextCraft official test split (100 tasks), avg of 4 passes, temperature 0.4, 512 tokens/turn, 30 turns, thinking off. Success: not evaluated. Reference points on the same protocol: base Qwen3-1.7B 23.00%, Qwen3-32B itself 85.50%.
Prompt format
ReAct, multi-turn chat: the environment's AgentGym instruction as the first user turn, a canned assistant acknowledgement, then one user
turn per observation; the model replies Thought:\n...\n\nAction:\n<one action>. Render with Qwen3's chat template and enable_thinking=False.