Qwen3-8B-Critic-SFT-Detailed-Prompt
An 8B critic from Steer, Don't Solve: Training Small Critic Models for Large Code Agents, trained on teacher critiques collected with the detailed prompt instead of the high-level one. It is the comparison arm of the prompt ablation (Table 4). The critic trained on high-level critiques is Qwen3-8B-Critic-SFT.
The two prompts differ in what the teacher is allowed to say. The high-level prompt asks for error detection plus short guidance and forbids full solutions. The detailed prompt lets the teacher propose concrete code-level fixes, so its critiques are longer and often contain code. The paper shows that agents copy this code (Table 5), and that a small critic trained on detailed critiques ends up weaker than one trained on high-level critiques.
All released models and datasets are listed on the organization page. Code and configs are in the critic-training repository.
Where it appears in the paper
| Paper location | Row label |
|---|---|
| Table 4, prompt ablation | + SFT (Detailed-Prompt) |
Original run name: qwen3-8b-full-sft-prm-r2egym-swebench-k5-opus-distill-32k-lr5e6-multiturn. In the repository the detailed prompt is the prm_issue_res config family and the high-level prompt is prm_issue_res_instructions.
Training data
code-critic-model/critic-sft-cwm-only-detailed-prompt, 3,135 examples.
- Tasks: 500 R2E-Gym instances from matplotlib, moto, and sympy, disjoint from SWE-bench Verified.
- Agent that produced the trajectories: CWM-32B, 500 trajectories.
- Teacher: Claude Opus 4.6, queried every 5 agent steps with the detailed prompt.
The same 500 tasks and agent as critic-sft-cwm-only; only the teacher prompt differs. Fewer examples survive the 32K token limit because detailed critiques are longer.
Training setup
Identical to Qwen3-8B-Critic-SFT apart from the data. Full-parameter SFT with LLaMA-Factory, config finetuning/qwen3_8b_critic_full_sft_l40s_train_multiturn_resumable.yaml.
| Setting | Value |
|---|---|
| Base model | Qwen/Qwen3-8B |
| Chat template | qwen3_nothink |
| Sequence length | 32,768 tokens |
| Loss | final critique turn only (mask_history: true) |
| Hardware | 8 x L40S, effective batch 8 |
| Optimizer | AdamW, lr 5e-6, cosine, warmup ratio 0.1 |
| Epochs | 3 |
| Precision | bf16 |
Results
Resolve rate on SWE-bench Verified, from Table 4 of the paper. Both critics were run with the same high-level, step-aware inference prompt; only the training critiques differ.
| Coding agent | No critic | + this critic (detailed training critiques) | + Qwen3-8B-Critic-SFT (high-level training critiques) |
|---|---|---|---|
| Qwen3-Next-80B-A3B | 20.0 | 24.6 | 25.2 |
| Qwen3-32B | 8.8 | 12.6 | 13.8 |
Note that the high-level critic was trained on the larger CWM plus Qwen3-Next corpus, so this comparison also reflects the corpus difference.
How to use
Same serving and launch procedure as Qwen3-8B-Critic-SFT: serve with vLLM in bf16 and pass the served name to scripts/run_critic_max150.sh with --prm. The served name must have an entry in mini-swe-agent/configs/litellm_model_registry.json; add one for this model if you use a new name.
Citation
@misc{gandhi2026steerdontsolvetraining,
title={Steer, Don't Solve: Training Small Critic Models for Large Code Agents},
author={Shubham Gandhi and Yiqing Xie and Atharva Naik and Ruichen Zhu and Carolyn Rose},
year={2026},
eprint={2606.21811},
archivePrefix={arXiv},
primaryClass={cs.SE},
url={https://arxiv.org/abs/2606.21811}
}