OneReason-8B-Pretrain (Competition)

Reasoning Foundation Models for Generative Recommendation

Paper | Model Zoo | Quick Start | Citation

Introduction

OneReason is a recommendation foundation model that connects large language models with generative recommender systems. It represents items as compact itemic tokens and trains the model to align itemic-token semantics with natural language, user behavior, and recommendation-oriented reasoning traces.

The OneReason training stack contains three stages:

• Pre-training: builds itemic-token perception through four-granularity itemic-text alignment data, covering token-, item-, relational-, and user-level signals.
• Supervised Fine-Tuning (SFT): teaches recommendation cognition with coarse-to-fine Chain-of-Thought (CoT) traces over user profiles, behavior histories, and itemic-token evidence.
• Reinforcement Learning (RL): uses a specialize-then-unify recipe to improve thinking-mode recommendation while balancing performance across multiple recommendation domains.

This repository releases the OneReason-8B-Pretrain (Competition) checkpoint — the 8B foundation checkpoint after itemic-text alignment pre-training, prepared for the OneReason competition track. The 0.8B pretrain checkpoint is released in a sibling repository.

News

• [2026.07] OneReason-8B-Pretrain (Competition) checkpoint is released.
• [2026.06] OneReason-0.8B Pretrain checkpoint is released.
• Coming soon: OneReason-0.8B / 8B SFT checkpoints.
• Coming soon: OneReason-0.8B / 8B RL checkpoints.

Model Zoo

Method Overview

Itemic Tokens

OneReason represents each item with one domain-aware begin token and three hierarchical sub-tokens:

<|domain_begin|><s_a_xxxx><s_b_xxxx><s_c_xxxx>

Supported recommendation domains include:

Each itemic token sequence is produced by a three-layer codebook, where each layer contains 8192 codes. The released checkpoint can process these itemic-token strings through its tokenizer. Mapping raw items to itemic tokens, or mapping generated itemic tokens back to real item IDs, requires the corresponding itemic tokenizer and item catalog.

Pre-training Data Design

OneReason pre-training uses 578B tokens to align itemic-token and text-token semantic spaces. The recommendation part follows a four-granularity corpus design:

• Token granularity: aligns individual and compositional sub-token semantics.
• Item granularity: aligns complete itemic patterns with natural-language captions and multi-perspective item QA.
• Relational granularity: injects item-to-item collaborative relations with natural-language transition explanations.
• User granularity: models user behavior sequences with domain-grouped and chronologically interleaved itemic-text formats.

General-domain text and multimodal corpora are mixed in to preserve instruction-following, reasoning, code, math, and broad semantic capabilities while injecting recommendation-specific knowledge.

Training Recipe

The pre-training recipe contains three stages:

OneReason-Bench

OneReason is evaluated with OneReason-Bench, a reasoning-oriented recommendation benchmark organized into four layers:

Performance

The released OneReason-8B-Pretrain (Competition) checkpoint is the 8B foundation checkpoint before SFT/RL. It provides stronger itemic-token perception and broader recommendation coverage than the 0.8B release, and serves as a strong initialization for downstream recommendation tuning.

The tables below report the full OneReason-8B system results from the technical report. The pretrain checkpoint itself is evaluated on R0 perception and grounding tasks; the SFT/RL rows are included to show the full system performance that this pretrain checkpoint enables.

Results on Cross-Domain Recommendation

Cross-domain recommendation results are reported in percentage. Best results are bolded; second-best results are underlined.

Results on R0-R2 Reasoning Tasks

R0-R2 results on OneReason-Bench are reported in percentage. For R0 tasks, results are macro-averaged over all domains. Grounding is reported by Pass@64.

Quick Start

Install dependencies:

pip install "transformers>=4.51.0" accelerate safetensors torch

Load the model:

from transformers import AutoModelForCausalLM, AutoTokenizer

model_name = "OpenOneRec/OneReason-8B-pretrain-competition"  # or the local path to this repository

tokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    model_name,
    torch_dtype="auto",
    device_map="auto",
    trust_remote_code=True,
)

Item Understanding Example

prompt = "<|prod_begin|><s_a_1183><s_b_746><s_c_5290>，这个商品卖的是什么？ /no_think"

messages = [{"role": "user", "content": prompt}]
text = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
)

inputs = tokenizer([text], return_tensors="pt").to(model.device)
outputs = model.generate(
    **inputs,
    max_new_tokens=512,
    do_sample=True,
    top_p=0.95,
    temperature=0.7,
)

response_ids = outputs[0][len(inputs.input_ids[0]):]
print(tokenizer.decode(response_ids, skip_special_tokens=True))

Expected response:

该商品是一款家居生活厨房小工具，一级类目是家居生活，二级类目是厨房小工具，三级类目是其他小工具。加厚食品级透明塑料材质，高透明度便于观察，耐高温无异味，可反复使用，适用于烘焙、饮品等场景。价格区间为5-10元。无品牌。

Itemic Pattern Grounding Example

prompt = (
    "根据描述生成短视频token：一个关于超市货架上的方便面种类及消费者在选择时遇到的困惑的视频。"
    "这段视频记录了一家超市的方便面货架，展示了各种不同口味的方便面。/no_think"
)

messages = [{"role": "user", "content": prompt}]
text = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
)

inputs = tokenizer([text], return_tensors="pt").to(model.device)
outputs = model.generate(
    **inputs,
    max_new_tokens=64,
    do_sample=False,
)

response_ids = outputs[0][len(inputs.input_ids[0]):]
print(tokenizer.decode(response_ids, skip_special_tokens=True))

Expected response:

<|video_begin|><s_a_5820><s_b_908><s_c_1352>

Recommendation-Style Example

The following prompt format illustrates how user profiles and behavior histories can be used for recommendation. For production deployment, you should use your own itemic tokenizer, item catalog, user behavior schema, and candidate decoding strategy.

prompt = (
    "参考以下用户信息：41-49 岁河北男性用户偏好短剧中的复仇商战题材，热衷于象棋、民族风情及民生资讯类短视频，"
    "常关注憨豆等直播内容并倾向于消费休闲模拟经营游戏。"
    "这个用户看过<|ad_begin|><s_a_7939><s_b_6234><s_c_4978>, "
    "<|ad_begin|><s_a_5673><s_b_6234><s_c_1614>, "
    "<|ad_begin|><s_a_3578><s_b_3009><s_c_3363>这些广告，"
    "该用户最近可能感兴趣的视频有哪些？ /no_think"
)

messages = [{"role": "user", "content": prompt}]
text = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
)

inputs = tokenizer([text], return_tensors="pt").to(model.device)
outputs = model.generate(
    **inputs,
    max_new_tokens=256,
    do_sample=True,
    top_p=0.95,
    temperature=0.7,
)

response_ids = outputs[0][len(inputs.input_ids[0]):]
print(tokenizer.decode(response_ids, skip_special_tokens=True))

Expected response (example output):

[
  '<|video_begin|><s_a_3801><s_b_7498><s_c_853>',
  '<|video_begin|><s_a_4615><s_b_4033><s_c_1014>',
  '<|video_begin|><s_a_7385><s_b_800><s_c_4636>',
  ...
  '<|video_begin|><s_a_4646><s_b_7261><s_c_853>'
]

Intended Use

The OneReason-8B-Pretrain (Competition) checkpoint is intended for:

• Research on generative recommendation and recommendation foundation models at the 8B scale.
• Continued pre-training or SFT on new recommendation domains.
• Itemic-token perception studies, including item understanding and itemic-token grounding.
• Building downstream recommendation models that combine user profiles, behavior histories, and itemic-token representations.
• Competition baselines that require a strong, open recommendation foundation checkpoint.

For best recommendation reasoning performance, we recommend using future SFT/RL checkpoints once released, or fine-tuning this pretrain checkpoint on task-specific supervised data.

Limitations

• This release is a pre-training checkpoint, not the final SFT/RL reasoning model.
• Direct recommendation quality depends on the itemic tokenizer, item catalog, user history format, and decoding strategy.
• Generated itemic tokens must be validated against the target item catalog before being used as item IDs.
• The model may generate invalid, stale, or unsupported itemic-token sequences if the prompt distribution differs significantly from training data.
• The 8B checkpoint requires more memory and compute than the 0.8B release; make sure your environment has sufficient GPU resources before loading.
• The checkpoint is released for research and competition use, and should not be used for high-stakes personalization without careful evaluation, filtering, and privacy review.

Citation

If you find OneReason useful, please cite our technical report. The official BibTeX will be updated after the report is publicly available.

@article{onereason2026,
  title   = {OneReason Technical Report},
  author  = {OneRec Team},
  journal = {Technical Report},
  year    = {2026}
}

License

Please refer to the license file in this repository for the terms governing the model weights and any accompanying code or assets.

Acknowledgements

OneReason builds on the open-source LLM and recommendation ecosystem. We thank the Qwen, OpenOneRec, PyTorch, Transformers, and distributed training communities for their foundational contributions.