KBevo-Qwen3-1.7B-GRPO
Final KBevo GRPO checkpoint (Qwen3-1.7B, step 500). Accompanies Co-Evolving Structured Knowledge and Reasoning in Language Models, published at the Conference on Language Modeling (COLM 2026).
KBevo: Structured knowledge + reasoning in two phases
KBevo teaches a single language model to answer multi-hop questions by first building a structured knowledge base and then reasoning over it:
- Knowledge-base construction. Given a set of passages, the model extracts a compact structured KB of typed
(entity, relation, value)triples. Inverse relations are added so the KB can be traversed in either direction. - Question answering through structured KB retrieval. For each question, the model issues typed KB lookups instead of re-reading raw text. Answers are grounded in retrieved triples rather than in parametric memory alone.
The two phases share weights and are trained jointly. SFT teaches the base extraction / lookup behaviour on synthesised two-phase rollouts. GRPO further optimises the joint policy on end-to-end HotpotQA answer-F1 rewards.
This repository
This repository contains the final KBevo GRPO checkpoint for Qwen3-1.7B (step 500). It is the checkpoint used for the paper's evaluation numbers below and is the recommended model for inference / evaluation.
Its SFT starting point is available at kilian-group/KBevo-Qwen3-1.7B-SFT.
All KBevo checkpoints
| Model | Repository |
|---|---|
| KBevo-Qwen3-1.7B SFT | kilian-group/KBevo-Qwen3-1.7B-SFT |
| KBevo-Qwen3-1.7B GRPO | kilian-group/KBevo-Qwen3-1.7B-GRPO |
| KBevo-Qwen3-4B SFT | kilian-group/KBevo-Qwen3-4B-SFT |
| KBevo-Qwen3-4B GRPO | kilian-group/KBevo-Qwen3-4B-GRPO |
Full code, training, and evaluation scripts: https://github.com/kilian-group/KBevo
Loading
from transformers import AutoModelForCausalLM, AutoTokenizer
repo_id = "kilian-group/KBevo-Qwen3-1.7B-GRPO"
tokenizer = AutoTokenizer.from_pretrained(repo_id)
model = AutoModelForCausalLM.from_pretrained(repo_id, dtype="bfloat16")
For the two-phase KBevo inference agent (KB construction → structured QA), please use the reference implementation from the KBevo GitHub repository. See scripts/eval_kbevo.sh.
Results
Exact-match on the four benchmarks reported in the paper:
| Model | HotpotQA | MuSiQue | 2WikiMultiHopQA | PopQA | Average |
|---|---|---|---|---|---|
| KBevo-Qwen3-1.7B-SFT | 29.5 | 13.3 | 44.9 | 57.0 | 36.2 |
| KBevo-Qwen3-1.7B-GRPO | 37.3 | 18.6 | 47.8 | 61.4 | 41.3 |
| KBevo-Qwen3-4B-SFT | 35.6 | 16.7 | 40.7 | 54.3 | 36.8 |
| KBevo-Qwen3-4B-GRPO | 46.1 | 26.4 | 51.4 | 62.5 | 46.6 |
Training
Stage: Two-phase GRPO on HotpotQA, initialised from kilian-group/KBevo-Qwen3-1.7B-SFT (SFT checkpoint at step 368).
Hyperparameters:
| Setting | Value |
|---|---|
| Underlying pretrained model | Qwen/Qwen3-1.7B |
| GRPO initialization | KBevo Qwen3-1.7B SFT checkpoint at step 368 |
| Training dataset | HotpotQA |
| Training examples | 7,000 |
| Evaluation examples during training | 100 |
| Objective | GRPO |
| Reward | Answer F1 |
| Learning rate | 5e-6 |
| Training steps | 500 |
| LR scheduler | Cosine |
| Warmup ratio | 0.1 |
| Maximum gradient norm | 1.0 |
| KL coefficient β | 0.0 |
| Per-device training batch size | 16 |
| Per-device evaluation batch size | 32 |
| Gradient accumulation steps | 8 |
| Effective training batch size | 512 |
| Number of GPUs | 4 × B200 |
| Maximum vLLM model length | 4,096 |
| Maximum completion length | 1,024 |
| Sampling temperature | 1.0 |
| Sampling top-p | 0.95 |
| Sampling top-k | 4 |
| Two-phase training | Enabled |
| Phase-1 KB rollouts per question, K | 4 |
| Phase-2 QA rollouts per KB, M | 8 |
| Total Phase-2 QA rollouts per question | 32 = 4 × 8 |
| Total generations per question | 36 = 4 KB + 32 QA |
| Phase-1 prompt type | SFT |
| Phase-1 database weight mode | Count |
| Retrieval similarity threshold | 0.6 |
| Retrieval top-k | 4 |
| Inverse relations | Enabled |
| Adaptive retrieval k | Disabled |
| Runtime precision | BF16 |
| Generation backend | vLLM |
| Gradient checkpointing | Enabled |
| vLLM colocation | Enabled |
| vLLM GPU memory utilization | 0.4 |
| Logging interval | 5 steps |
| Evaluation interval | 100 steps |
| Checkpoint saving interval | 25 steps |
| Maximum saved checkpoints | 5 |
Intended use
This checkpoint is intended for running and evaluating KBevo's two-phase knowledge construction and reasoning policy. It can be used with the official KBevo codebase to reproduce the reported results or as a starting point for further research and fine-tuning.
Citation
@inproceedings{Noonan2026:co-evolving,
title = {Co-Evolving Structured Knowledge and Reasoning in Language Models},
author = {Ryan Thomas Noonan and Linxi Zhao and Menghan Xu and Akanksha Sarkar and Mihir Mishra and Dongyoung Go and Kilian Q. Weinberger and Yoav Artzi and Jennifer J. Sun},
booktitle = {Proceedings of the Conference on Language Modeling},
year = {2026},
url = {https://arxiv.org/abs/2608.26386}
}