GSM8K CoT fine-tune of Llama-3.2-3B-Instruct
Built with Llama. Supervised chain-of-thought fine-tune of meta-llama/Llama-3.2-3B-Instruct on GSM8K.
This model serves as the Stage-1 CoT baseline/initialization checkpoint for LOTUS (Looped Transformers with parallel supervision on latents), as described in the paper Bridging the Gap Between Latent and Explicit Reasoning with Looped Transformers.
Links
- Paper: arXiv:2606.31779
- Code: GitHub - yingfan-bot/lotus
Usage
from transformers import AutoModelForCausalLM, AutoTokenizer
model = AutoModelForCausalLM.from_pretrained("yingfanbot/gsm-cot-llama3b")
tokenizer = AutoTokenizer.from_pretrained("yingfanbot/gsm-cot-llama3b")
Citation
@misc{fan2026bridginggaplatentexplicit,
title={Bridging the Gap Between Latent and Explicit Reasoning with Looped Transformers},
author={Ying Fan and Anej Svete and Kangwook Lee},
year={2026},
eprint={2606.31779},
archivePrefix={arXiv},
primaryClass={cs.LG},
url={https://arxiv.org/abs/2606.31779},
}