qwen3-4b-thinking-thinkprune-b4000
Paper: Efficient Reasoning Training Does Not Always Harm CoT Faithfulness and Monitorability
Collection: Efficient Reasoning: CoT Faithfulness & Monitorability
Method: ThinkPrune (Hou et al., 2026)
Hyperparameters
| Base model | Qwen/Qwen3-4B-Thinking-2507 |
| Method | ThinkPrune |
| Released as | merged full model, step 200 |
| Completion budget L | 4,000 tokens |
| LoRA | rank 16, alpha 32, dropout 0, all attention and MLP projections |
| Optimiser | AdamW, learning rate 1e-5, KL coefficient 0 |
| Batch | 32 prompts x 16 generations |
| Sampling | temperature 0.8, top-p 0.95 |
| Data | numina_amc_aime (PRIME Eurus-2-RL-Data), ~2,223 prompts |
| Checkpoint selection | AIME22, every 50 steps |
| Hardware | 2x NVIDIA GH200 |
Merged from the step-200 LoRA adapter (Samll/qwen3-4b-thinkprune-2gpu-step200).
Usage
from transformers import AutoModelForCausalLM, AutoTokenizer
repo = "Samll/qwen3-4b-thinking-thinkprune-b4000"
tok = AutoTokenizer.from_pretrained(repo)
model = AutoModelForCausalLM.from_pretrained(repo, torch_dtype="auto", device_map="auto")