epe-1p-3b-llama3arch-smollm2tok-500B-40n-2048sl-960gbsz-no_bce-refl_midtrain_from_epe
Converted Hugging Face base checkpoint from the Model Raising pretraining run.
Details
- Architecture:
LlamaForCausalLM - Base model size:
3B - Precision on disk:
bfloat16 - Source Megatron checkpoint iteration:
72895 - Model kind:
epe - Config vocab size:
49280
Tokenizer
Use the bundled tokenizer from this repository.
This EPE checkpoint uses the extended SmolLM2 tokenizer with <assistant> and 35 <charter_X.Y> tokens. Two named chat templates are available:
| Name | Assistant turn start |
|---|---|
default | `< |
epe | `< |