Qwen3-4B QuestA Non-MCQ E-step (Step 50)
This is the Hugging Face-format checkpoint at step 50 of a research E-step
experiment initialized from Qwen/Qwen3-4B-Instruct-2507.
Training setup
- Source dataset:
YYYYYYibo/QuestA-OpenR1-Math-220k-Joined - Training subset: 5,897 deduplicated non-multiple-choice problems
- Teacher context: original problem plus the dataset
solution - Frozen student context: original problem without the solution
- Objective: sampled teacher-penalty PPO signal corresponding to
KL(teacher || frozen student), plus sequence-level correctness rewards - Correct reward:
+20 - Incorrect penalty:
-20 - Length-truncation penalty:
-20 - Training steps: 50
- Global batch size: 256
- PPO micro-batch size per GPU: 2
- Learning rate:
1e-6 - Maximum prompt length: 4,096 tokens
- Maximum training response length: 8,192 tokens
- Hardware: 2x NVIDIA H200 NVL
The saved FSDP actor checkpoint was merged into standard Transformers safetensors before upload.
In-sample diagnostic evaluation
The following diagnostic used 256 randomly selected problems from the training dataset, one rollout per problem, and a 16,384-token response limit. These numbers are in-sample diagnostics and are not held-out generalization results.
| Condition | Math-Verify accuracy | Accuracy with truncation counted wrong | Truncation rate |
|---|---|---|---|
| Base, no solution | 56.25% | 55.08% | 5.47% |
| Base + solution | 92.19% | 91.41% | 1.17% |
| This checkpoint + solution | 78.91% | 77.34% | 2.73% |
This checkpoint is an experimental research artifact and should be evaluated on a genuinely held-out dataset before drawing conclusions about generalization.