EXAONE 3.5 7.8B VeriReason reproduced (full fine-tuning)
This is the best checkpoint from a full SFT run of LGAI-EXAONE/EXAONE-3.5-7.8B-Instruct on Jongbin-kr/VeriReason-reasoning-reproduced-1513.
Best checkpoint
| Item | Value |
|---|---|
| Selected checkpoint | checkpoint-300 |
| Selection metric | validation eval_loss (lower is better) |
| Best eval loss | 0.45221662521362305 |
| Training completed | 4 epochs / 760 optimizer steps |
| Final eval loss | 0.5761 |
| Training loss | 0.08864 |
The uploaded model.safetensors is byte-identical to local checkpoint-300/model.safetensors: SHA-256 f34495277ed0c864c9ba3bf5a7648b8333746ae9c9bf06b38d0e3bc922538c60.
The Slurm job was marked OUT_OF_MEMORY only after training and final evaluation completed, while DeepSpeed reloaded the best checkpoint for load_best_model_at_end. This did not affect the selected checkpoint or the model artifact above.
Training configuration
- Method: full fine-tuning SFT with DeepSpeed ZeRO-2 (4 ranks), bf16
- Loss: assistant completion tokens only, including reasoning and answer
- Max sequence length: 4096; packing disabled; gradient checkpointing enabled
- Learning rate:
1e-5, cosine schedule, warmup ratio0.1 - Per-device batch size: 1; gradient accumulation: 2; effective global batch: 8
- Seed: 42
- Evaluation: 189-example validation split, every 100 steps
Quick start
from transformers import pipeline
generator = pipeline(
"text-generation",
model="Jongbin-kr/exaone-7b-verireason-reproduced-1513-fullft-epoch4",
device="cuda",
)
output = generator(
[{"role": "user", "content": "Explain your reasoning, then answer the question."}],
max_new_tokens=128,
return_full_text=False,
)[0]
print(output["generated_text"])