LFM2.5-8B-A1B-KO-CPT-Repair-SFT
This is a diagnostic repair-SFT checkpoint trained from
LLM-OS-Models/LFM2.5-8B-A1B-KO-CPT-FULL.
It is published for reproducibility, not as the recommended benchmark model.
The current recommendation is still the KO-CPT checkpoint. This repair SFT did not recover the CPT public benchmark profile.
- SFT GitHub: https://github.com/gyunggyung/LFM25-KO-SFT
- CPT GitHub: https://github.com/gyunggyung/LFM25-KO-CPT
- CPT model: https://huggingface.co/LLM-OS-Models/LFM2.5-8B-A1B-KO-CPT-FULL
Training Data
The run used a small answer-format repair mixture instead of the previous large general SFT line.
| item | value |
|---|---|
| prepared dataset | /home/work/.data/lfm2_ko_sft/prepared/repair_cpt/20260630_cpt_mcqa_repair_4k/lfm_chat_4k |
| samples | 188,493 |
| LFM tokens | 131,607,379 |
| max sequence length | 4,096 |
| epochs | 1 |
| learning rate | 1e-6 |
| global effective batch | 128 |
| planned optimizer steps | 1,473 |
The mixture focused on Korean MCQA answer formatting, KoTSQA train evidence QA, finance/Text2SQL preservation, SWE/coding preservation, and compact reasoning preservation. It intentionally avoided continuing from the failed Stage2/Stage3 SFT checkpoints.
Gate Evaluation
Evaluation root:
/home/work/.data/lfm2_ko_sft/eval/repair_sft_gate_20260630T1306KST
Base/CPT reference values are copied from the KO-CPT model card where available. Some metrics differ from this gate run, so rows marked with a metric mismatch should be read directionally rather than as strict apples-to-apples comparisons.
| task | Base | KO-CPT | Repair-SFT | repair metric | verdict |
|---|---|---|---|---|---|
| BoolQ | 0.6544 | 0.7902 | 0.663303 | acc,none | above Base, far below CPT |
| ARC-Challenge | 0.3771 | 0.4241 | 0.211604 | acc,none; CPT uses acc_norm | below Base/CPT |
| GSM8K | 0.4845 | 0.5701 | 0.329795 | strict-match; CPT uses flexible-extract | below Base/CPT |
| IFEval | 0.2921 | 0.3216 | 0.181146 | strict prompt acc; CPT uses loose prompt acc | below Base/CPT |
| Global MMLU KO jurisprudence | 0.2870 | 0.2685 | 0.250000 | acc,none | below Base/CPT |
| KMMLU direct hard | 0.2015 | 0.1720 | 0.102339 | exact_match,none; CPT card uses acc,none | below Base/CPT |
| MMLU-ProX Lite KO | 0.2585 | 0.1667 | 0.091837 | exact_match,custom-extract | below Base/CPT |
Interpretation
The repair attempt did not solve the regression seen in the earlier KO-SFT line. The likely failure mode is still answer distribution drift: response-only chat SFT changed the model toward verbose assistant behavior and away from short exact-answer extraction required by public MCQA tasks.
Final lesson: CPT improved Korean/domain knowledge and parts of the benchmark profile, but broad SFT did not reliably repair the MCQA and short-answer regression. This checkpoint is evidence that simply adding more SFT, even with a repair-oriented mix, can preserve the wrong verbose assistant distribution instead of restoring concise option selection.
For public benchmark reporting, prefer the KO-CPT checkpoint. If another repair attempt is made, it should be much smaller, use a lower learning rate, and stop early based on 100/300/500-step gate evaluations.