zimplex/olmo3-7b-think-sft-eosfix-16k-3ep-hyp
Final EOS-fixed Thinking-SFT rerun of allenai/Olmo-3-1025-7B using a
hyperbolic (Lorentzian) output head. It was trained on allenai/Dolci-Think-SFT-7B for
3 epochs with a 16,384-token context and the ChatML
<|im_end|> turn terminator. Generation stops on either <|im_end|> or
<|endoftext|>.
Training provenance
- Source commit:
299291204767d845d0534b9d0a89697e73c62d41 - Base revision:
a81bae42db3975be1671e27b9c9a56da1a9f980f - Global batch size: 64 sequences
- Learning rate: 8e-5
- Training hardware: 4 nodes / 32 B200 GPUs
- W&B training run: https://wandb.ai/vista-01/hyp-think-sft/runs/o8tkf98o
Reasoning evaluation
Greedy decoding, up to 4,096 generated tokens, using the repository's <think> template.
| Task | Accuracy | Terminated | Repetition rate |
|---|---|---|---|
| math | 0.548 | 0.490 | 0.136 |
| gsm8k | 0.880 | 0.908 | 0.127 |
| mmlu_pro | 0.374 | 0.374 | 0.211 |
| gpqa_diamond | 0.293 | 0.192 | 0.320 |
W&B evaluation run: https://wandb.ai/vista-01/hyp-think-sft/runs/caeugb6s
Loading
This checkpoint has a custom hyperbolic output head. A plain
AutoModelForCausalLM.from_pretrained(...) loads the frozen Euclidean lm_head and is not
equivalent. Use hyp_head_state.pt with the loader/evaluation code from source commit
299291204767d845d0534b9d0a89697e73c62d41 (for example scripts/eval_think.py). The head archive is authoritative for
the projection, adapter, curvature and learned scales.
Reproducibility files
training_provenance.json records the source/base revisions and recipe.
eval_metrics.json contains the portable evaluation results. Cluster-local paths and
the internal .save_complete file are intentionally not published.