Reasoning Cache (RCT-4B)
This repository contains the RCT-4B model checkpoint, a 4B parameter Large Language Model (LLM) trained using Reasoning Cache (RC), as introduced in the paper Reasoning Cache: Continual Improvement Over Long Horizons via Short-Horizon RL.
Model Description
Reasoning Cache (RC) is an iterative decoding algorithm that replaces standard autoregressive decoding during both training and inference. It exploits an asymmetry between the response generation and summarization capabilities of LLMs to construct reasoning chains that consistently improve across iterations.
Models trained to use RC can extrapolate and continually improve over reasoning horizons significantly longer than those seen during training. Empirically, RCT-4B demonstrates substantial performance gains on challenging benchmarks like HMMT 2025, outperforming both comparably sized models and many larger reasoning LLMs by effectively scaling test-time compute.
- Developed by: Ian Wu, Yuxiao Qu, Amrith Setlur, Aviral Kumar
- Paper: Reasoning Cache: Continual Improvement Over Long Horizons via Short-Horizon RL
- Repository: GitHub - IanYHWu/rc
Usage
This model can be loaded using the Hugging Face transformers library for standard model and tokenizer operations. However, to leverage the iterative RC-decoding algorithm and achieve the continual improvement and extrapolation capabilities described in the paper, specific inference logic must be applied.
For detailed instructions on using RC-decoding for inference (which currently supports vLLM) and for the training code (which requires verl), please refer to the official GitHub repository.
# Example of loading the model with transformers (RC-decoding logic not included here)
from transformers import AutoTokenizer, AutoModelForCausalLM
model_name = "YOUR_REPO_ID_HERE" # Replace with the actual repository ID
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name)
# For RC-decoding, please refer to the official GitHub repository's `inference` section.
Citation
If you find this work useful, please cite the original paper:
@article{wu2026reasoning,
title={Reasoning Cache: Continual Improvement Over Long Horizons via Short-Horizon RL},
author={Wu, Ian and Qu, Yuxiao and Setlur, Amrith and Kumar, Aviral},
journal={arXiv preprint arXiv:2602.03773},
year={2026}
}