Qwen3-1.7B · TextCraft · online ROSE 5+5 with a Qwen3-32B teacher — step 66 of 66

Student: Qwen/Qwen3-1.7B. Part of a study comparing teacher-trajectory SFT, online multi-turn ROSE and on-policy distillation for multi-turn agents.

Training

Online multi-turn ROSE: the student plays the first 5 turns in a live environment, the teacher continues the same environment for up to 5 turns, and only the teacher's turns are cross-entropy targets. 32 episodes per step, AdamW constant lr 1e-5, bf16, thinking off, 512 tokens per turn. Teacher: Qwen3-32B (bf16, local). 374 training tasks, 6 epochs = 66 steps.

Evaluation

TextCraft official test split (100 tasks), avg of 4 passes, temperature 0.4, 512 tokens/turn, 30 turns, thinking off. Success: 54.75%. Reference points on the same protocol: base Qwen3-1.7B 23.00%, Qwen3-32B itself 85.50%.

Notes

Trained with match_chat_template off: every observation follows a generated turn's <|im_end|> without the template's newline, a small train/eval mismatch that later runs fixed.

Prompt format

ReAct, multi-turn chat: the environment's AgentGym instruction as the first user turn, a canned assistant acknowledgement, then one user turn per observation; the model replies Thought:\n...\n\nAction:\n<one action>. Render with Qwen3's chat template and enable_thinking=False.