olmo3-7b-think-thinkprune-b4000
Paper: Efficient Reasoning Training Does Not Always Harm CoT Faithfulness and Monitorability
Collection: Efficient Reasoning: CoT Faithfulness & Monitorability
Method: ThinkPrune (Hou et al., 2026)
Hyperparameters
| Base model | allenai/Olmo-3-7B-Think |
| Method | ThinkPrune |
| Released as | merged full model, step 200 |
| Completion budget L | 4,000 tokens |
| LoRA | rank 16, alpha 32, dropout 0, all attention and MLP projections |
| Optimiser | AdamW, learning rate 1e-5, KL coefficient 0 |
| Batch | 32 prompts x 16 generations |
| Sampling | temperature 0.8, top-p 0.95 |
| Data | numina_amc_aime (PRIME Eurus-2-RL-Data), ~2,223 prompts |
| Checkpoint selection | AIME22, every 50 steps |
| Hardware | 2x NVIDIA GH200 |
Usage
from transformers import AutoModelForCausalLM, AutoTokenizer
repo = "Samll/olmo3-7b-think-thinkprune-b4000"
tok = AutoTokenizer.from_pretrained(repo)
model = AutoModelForCausalLM.from_pretrained(repo, torch_dtype="auto", device_map="auto")