OctoThinker-8B-Long-Base GGUF
GGUF quantizations of OctoThinker/OctoThinker-8B-Long-Base.
OctoThinker-8B-Long-Base is a mid-trained language model built on the Llama-3 family, designed to create a reinforcement learning–friendly base language model. It is trained using the Stable-then-Decay two-stage mid-training paradigm on large-scale mathematical datasets (MegaMath-Web-Pro-Max) and diversified Long Chain-of-Thought data, ensuring rapid convergence and improved RL stability.
For more details, see the paper.
Available Quantizations
| File | Bits | Size (approx) | Use Case |
|---|---|---|---|
| model_f16.gguf | 16 | ~16.1 GB | Maximum quality, reference |
| model_q8_0.gguf | 8 | ~8.5 GB | Best quality, if VRAM allows |
| model_q6_k.gguf | 6 | ~6.6 GB | Great quality |
| model_q5_k_m.gguf | 5 | ~5.7 GB | Balanced quality/size |
| model_q5_k_s.gguf | 5 | ~5.5 GB | Slightly smaller Q5 |
| model_q4_k_m.gguf | 4 | ~4.8 GB | Good quality, recommended |
| model_q4_k_s.gguf | 4 | ~4.6 GB | Smaller Q4 |
| model_q3_k_l.gguf | 3 | ~4.0 GB | Low VRAM, decent quality |
| model_q3_k_m.gguf | 3 | ~3.7 GB | Lower VRAM |
| model_q3_k_s.gguf | 3 | ~3.5 GB | Minimum Q3 |
| model_q2_k.gguf | 2 | ~3.0 GB | Very low VRAM, reduced quality |
VRAM Requirements
| Quantization | VRAM (approx) |
|---|---|
| F16 | ~17 GB |
| Q8_0 | ~9.5 GB |
| Q6_K | ~7.5 GB |
| Q5_K_M | ~6.5 GB |
| Q4_K_M | ~5.5 GB |
| Q3_K_M | ~4.5 GB |
| Q2_K | ~3.5 GB |
Usage
llama.cpp
./llama-cli -m model_q4_k_m.gguf -p "Your prompt here" -n 512
llama-cpp-python
from llama_cpp import Llama
llm = Llama(model_path="model_q4_k_m.gguf", n_ctx=8192)
output = llm("Your prompt here", max_tokens=512)
print(output["choices"][0]["text"])
LM Studio
Download the desired .gguf file and load it directly in LM Studio.
Ollama
ollama run hf.co/tinyopsec/OctoThinker-8B-Long-Base-GGUF
Citation
@article{wang2025octothinker,
title={OctoThinker: Mid-training Incentivizes Reinforcement Learning Scaling},
author={Wang, Zengzhi and Zhou, Fan and Li, Xuefeng and Liu, Pengfei},
year={2025},
journal={arXiv preprint arXiv:2506.20512},
note={Preprint}
}