Nemotron-Research-Reasoning-Qwen-1.5B

News

• [2025-11-20] Nemotron-Research-Reasoning-Qwen-1.5B-BroRL is released.
• [2025-08-11] ProRL V2 blog post is released: ProRL V2 - Prolonged Training Validates RL Scaling Laws.
• [2025-07-23] Nemotron-Research-Reasoning-Qwen-1.5B-v2 is released.
• [2025-05-29] Nemotron-Research-Reasoning-Qwen-1.5B is released.

Introduction

Nemotron-Research-Reasoning-Qwen-1.5B is the world’s leading 1.5B open-weight model for complex reasoning tasks such as mathematical problems, coding challenges, scientific questions, and logic puzzles. It is trained using the ProRL algorithm on a diverse and comprehensive set of datasets. Our model has achieved impressive results, outperforming Deepseek’s 1.5B model by a large margin on a broad range of tasks, including math, coding, and GPQA.

This model is for research and development only.

ProRL: Prolonged Reinforcement Learning

ProRL is designed to enable extended RL training periods that facilitate deeper exploration of reasoning strategies. It enables more than 2k training steps and scale the training data across diverse tasks—from traditional math and code tasks to STEM problems, logical puzzles, and instruction following, which, we hypothesize, are crucial for generalization. Based on Group Relative Policy Optimization (GRPO), ProRL introduces three key techniques:

1. Mitigating Entropy Collapse
2. Decoupled clip and dynamic sampling policy optimization (DAPO)
3. KL regularization and reference policy reset

Using ProRL, we developed the world's best 1.5B reasoning model that significantly outperforms its base model, DeepSeek-R1-1.5B, and matches or even surpasses the performance of DeepSeek-R1-7B across a diverse range of benchmarks. Notably, compared to DeepSeek-R1-1.5B, we achieve average pass@1 improvements of 14.7% on math benchmarks, 13.9% on coding, 54.8% on logic puzzles, 25.1% on STEM reasoning, and 18.1% on instruction-following tasks.

Training Datasets

Evaluation Results

Table 1: Performance (pass@1) comparison for benchmarks across Math domain.

Table 2: Performance (pass@1) comparison across benchmarks for Code. We abbreviate benchmarks names for codecontests (cc), codeforces (cf), humanevalplus (human), and livecodebench (LCB).

Table 3: Performance comparison on STEM reasoning (GPQA Diamond), instruction following (IFEval), and logic puzzles (Reasoning Gym) tasks. We also present results on OOD tasks: acre, boxnet, and game_of_life_halting (game).

Nemotron-Research-Reasoning-Qwen-1.5B-v2

In the wake of the release of Nemotron-Research-Reasoning-Qwen-1.5B, we scaling the training steps from 2000 to 3000, resulting in Nemotron-Research-Reasoning-Qwen-1.5B-v2. Nemotron-Research-Reasoning-Qwen-1.5B-v2 builds on top of REINFORCE++-baseline with dynamic sampling and clip-higher, and proposes several critical enhancements such as periodically refreshing the reference model with the current best checkpoint and imposing the length penalty only in scheduled cycles. Together, these techniques allow model performance to continually improve with more RL training steps and expand LLMs' reasoning boundaries. Our latest checkpoint, Nemotron-Research-Reasoning-Qwen-1.5B-v2, trained for 3000 steps, sets a new state-of-the-art (SOTA) among 1.5B reasoning models.

For the Nemotron-Research-Reasoning-Qwen-1.5B-v2, you can use the following code to load the model:

from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("nvidia/Nemotron-Research-Reasoning-Qwen-1.5B")
model = AutoModelForCausalLM.from_pretrained("nvidia/Nemotron-Research-Reasoning-Qwen-1.5B")

For the original Nemotron-Research-Reasoning-Qwen-1.5B, you can use the following code to load the model:

from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("nvidia/Nemotron-Research-Reasoning-Qwen-1.5B", revision="v1")
model = AutoModelForCausalLM.from_pretrained("nvidia/Nemotron-Research-Reasoning-Qwen-1.5B", revision="v1")

BroRL

In BroRL, we continued training for 419 steps based on a nearly fully trained ProRLv2 checkpoint, increasing the number of samples per prompt from 16 to 512. We found that the improvement of BroRL over ProRLv2 was greater than that of ProRLv2 over ProRLv1.

Link to the BroRL 419 steps checkpoint

License/Terms of Use

cc-by-nc-4.0

Ethical Considerations

NVIDIA believes Trustworthy AI is a shared responsibility and we have established policies and practices to enable development for a wide array of AI applications. When downloaded or used in accordance with our terms of service, developers should work with their internal model team to ensure this model meets requirements for the relevant industry and use case and addresses unforeseen product misuse.

Please report security vulnerabilities or NVIDIA AI Concerns here.

Citation

If you find our dataset helpful, please cite the following ProRL paper and BroRL paper:

@article{liu2025prorl,
  author    = {Mingjie Liu, Shizhe Diao, Ximing Lu, Jian Hu, Xin Dong, Yejin Choi, Jan Kautz, Yi Dong},
  title={ProRL: Prolonged Reinforcement Learning Expands Reasoning Boundaries in Large Language Models}, 
  journal   = {arXiv preprint},
  year      = {2025},
  archivePrefix = {arXiv},
  primaryClass = {cs.CL},
  url={https://arxiv.org/abs/2505.24864}, 
}

@article{hu2025brorl,
  title={Brorl: Scaling reinforcement learning via broadened exploration},
  author={Hu, Jian and Liu, Mingjie and Lu, Ximing and Wu, Fang and Harchaoui, Zaid and Diao, Shizhe and Choi, Yejin and Molchanov, Pavlo and Yang, Jun and Kautz, Jan and others},
  journal={arXiv preprint arXiv:2510.01180},
  year={2025}
}