GSM8K CoT fine-tune of Llama-3.2-1B-Instruct
Built with Llama. Supervised chain-of-thought fine-tune of meta-llama/Llama-3.2-1B-Instruct on GSM8K.
This model serves as the Stage 1 supervised fine-tuning (SFT) baseline for LOTUS (Looped Transformers with parallel supervision on latents), as presented in the paper Bridging the Gap Between Latent and Explicit Reasoning with Looped Transformers.
- Paper: Bridging the Gap Between Latent and Explicit Reasoning with Looped Transformers
- Repository: GitHub - yingfan-bot/lotus
Usage
from transformers import AutoModelForCausalLM, AutoTokenizer
model = AutoModelForCausalLM.from_pretrained("yingfanbot/gsm-cot-llama1b")
tokenizer = AutoTokenizer.from_pretrained("yingfanbot/gsm-cot-llama1b")
Citation
If you use this model or code, please cite the paper:
@misc{fan2026bridginggaplatentexplicit,
title={Bridging the Gap Between Latent and Explicit Reasoning with Looped Transformers},
author={Ying Fan and Anej Svete and Kangwook Lee},
year={2026},
eprint={2606.31779},
archivePrefix={arXiv},
primaryClass={cs.LG},
url={https://arxiv.org/abs/2606.31779},
}