CodeScout-14B
📄 Paper • 💻 Code • 🤗 Collection
Strongest CodeScout model — open-source SOTA on SWE-Bench code localization.
CodeScout-14B is part of the CodeScout family of open-source RL-trained code search agents. CodeScout models achieve state-of-the-art repository-level code localization using nothing more than a standard Unix terminal — no static analysis, no repository graphs, no language-specific tooling.
Key Highlights
- Open-source SOTA on SWE-Bench Verified, Pro, and Lite for code localization
- Outperforms 2–18× larger base and post-trained LLMs across all benchmarks
- Surpasses GPT-5 and approaches Claude Sonnet 4.5 using RepoNavigator, despite using only a bash terminal
- Achieves 8–33% higher function-level F1 than Qwen3-32B (Thinking)
Results
Performance on SWE-Bench code localization (instance-averaged F1 scores):
| Benchmark | CodeScout-1.7B | CodeScout-4B | CodeScout-14B |
|---|---|---|---|
| SWE-Bench Verified — File F1 | 55.46 | 68.52 | 68.57 |
| SWE-Bench Verified — Func F1 | 28.22 | 36.78 | 40.32 |
| SWE-Bench Pro — File F1 | 40.96 | 51.77 | 53.63 |
| SWE-Bench Pro — Func F1 | 18.24 | 29.03 | 28.74 |
| SWE-Bench Lite — File F1 | 56.57 | 67.03 | 71.84 |
| SWE-Bench Lite — Func F1 | 27.07 | 39.87 | 44.43 |
Training
CodeScout-14B is trained directly from Qwen3-14B using GSPO (Group Sequence Policy Optimization) reinforcement learning.
- Training data: 9,600 instances from SWE-Smith (39K filtered, 128 repos)
- RL steps: 300
- Batch size: 32, with 4 rollouts per instance
- Max context length: 50K tokens (extended with YaRN)
- Max turns per episode: 4
- Reward: Multi-level F1 (file + module + function) with auxiliary turn-completion reward
- Hardware: 8×H100 GPUs
- Learning rate: 1e-6 (constant)
How It Works
CodeScout uses the OpenHands-Bash scaffold — an agent equipped with only a Terminal tool (supporting standard Unix commands like rg, find, grep, ls) and a LocalizationFinish tool for structured output submission. The agent iteratively navigates the repository to identify relevant files, classes, and functions related to a given issue.
The model is trained with GSPO (Group Sequence Policy Optimization) using multi-level F1 rewards at the file, module, and function level.
Intended Use
CodeScout-14B is designed for repository-level code localization: given a GitHub issue description and a code repository, it identifies the relevant files, classes, and functions that need to be modified. It is intended to be used as a localization subagent within larger coding agent pipelines.
Limitations
- Trained and evaluated exclusively on Python repositories
- Designed for code localization, not code editing or issue resolution
- Performance may vary on repositories significantly different from the training distribution
- Requires the OpenHands-Bash scaffold for optimal performance
Citation
@misc{sutawika2026codescouteffectiverecipereinforcement,
title={CodeScout: An Effective Recipe for Reinforcement Learning of Code Search Agents},
author={Lintang Sutawika and Aditya Bharat Soni and Bharath Sriraam R R and Apurva Gandhi and Taha Yassine and Sanidhya Vijayvargiya and Yuchen Li and Xuhui Zhou and Yilin Zhang and Leander Melroy Maben and Graham Neubig},
year={2026},
eprint={2603.17829},
archivePrefix={arXiv},
primaryClass={cs.SE},
url={https://arxiv.org/abs/2603.17829},
}