Qwen3.5-4B SWE-OPD baseline-simple (step 35)
On-policy distillation (OPD) checkpoint of Qwen3.5-4B (text) after 1 epoch / 35 steps on the tmax15k Harbor SWE set.
This is the merged Hugging Face export of the FSDP actor checkpoint:
swe_opd_baseline_simple / baseline_simple_1ep_bs12_lr5e7 / global_step_35
Training
| Item | Value |
|---|---|
| Base model | Qwen3.5-4B text (Qwen3_5ForCausalLM) |
| Method | On-policy distillation (OPD) |
| Dataset | tmax15k_top500_harbor_index30 |
| Steps | 35 (1 epoch) |
| Train batch size | 12 |
| Actor LR | 5e-7 |
| Max response length | 8192 |
| Distillation top-k | 8 |
| Student parallel | 12 GPU FSDP, DP=12 |
| Teacher | tmax-9b |
| Teacher probe / eval-turn mask | off (baseline-simple) |
Files
Merged from 12 FSDP shards (model_world_size_12_rank_*.pt) to a single bfloat16 model.safetensors.
Usage
Requires transformers>=5.6.0.
from transformers import AutoModelForCausalLM, AutoTokenizer
model_id = "StephYang/qwen3.5-4b-swe-opd-baseline-simple-step35"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, dtype="bfloat16", device_map="auto")