KBevo-Qwen3-4B-SFT
KBevo supervised fine-tuning (SFT) checkpoint for Qwen3-4B. Accompanies Co-Evolving Structured Knowledge and Reasoning in Language Models, published at the Conference on Language Modeling (COLM 2026).
KBevo: Structured knowledge + reasoning in two phases
KBevo teaches a single language model to answer multi-hop questions by first building a structured knowledge base and then reasoning over it:
- Knowledge-base construction. Given a set of passages, the model extracts a compact structured KB of typed
(entity, relation, value)triples. Inverse relations are added so the KB can be traversed in either direction. - Question answering through structured KB retrieval. For each question, the model issues typed KB lookups instead of re-reading raw text. Answers are grounded in retrieved triples rather than in parametric memory alone.
The two phases share weights and are trained jointly. SFT teaches the base extraction / lookup behaviour on synthesised two-phase rollouts. GRPO further optimises the joint policy on end-to-end HotpotQA answer-F1 rewards.
This repository
This repository contains the KBevo SFT checkpoint for Qwen3-4B. It is the starting point for the paper's KBevo-Qwen3-4B-GRPO run. Use it if you want the SFT-only KBevo behaviour or to reproduce the GRPO training pipeline.
For the paper's final KBevo checkpoint, see kilian-group/KBevo-Qwen3-4B-GRPO.
All KBevo checkpoints
| Model | Repository |
|---|---|
| KBevo-Qwen3-1.7B SFT | kilian-group/KBevo-Qwen3-1.7B-SFT |
| KBevo-Qwen3-1.7B GRPO | kilian-group/KBevo-Qwen3-1.7B-GRPO |
| KBevo-Qwen3-4B SFT | kilian-group/KBevo-Qwen3-4B-SFT |
| KBevo-Qwen3-4B GRPO | kilian-group/KBevo-Qwen3-4B-GRPO |
Full code, training, and evaluation scripts: https://github.com/kilian-group/KBevo
Loading
from transformers import AutoModelForCausalLM, AutoTokenizer
repo_id = "kilian-group/KBevo-Qwen3-4B-SFT"
tokenizer = AutoTokenizer.from_pretrained(repo_id)
model = AutoModelForCausalLM.from_pretrained(repo_id, dtype="bfloat16")
For the two-phase KBevo inference agent (KB construction → structured QA), please use the reference implementation from the KBevo GitHub repository. See scripts/eval_kbevo.sh.
Results
Exact-match on the four benchmarks reported in the paper:
| Model | HotpotQA | MuSiQue | 2WikiMultiHopQA | PopQA | Average |
|---|---|---|---|---|---|
| KBevo-Qwen3-1.7B-SFT | 29.5 | 13.3 | 44.9 | 57.0 | 36.2 |
| KBevo-Qwen3-1.7B-GRPO | 37.3 | 18.6 | 47.8 | 61.4 | 41.3 |
| KBevo-Qwen3-4B-SFT | 35.6 | 16.7 | 40.7 | 54.3 | 36.8 |
| KBevo-Qwen3-4B-GRPO | 46.1 | 26.4 | 51.4 | 62.5 | 46.6 |
Training
Stage: Supervised fine-tuning on synthesised two-phase rollouts over HotpotQA.
Hyperparameters:
| Setting | Value |
|---|---|
| Base model | Qwen/Qwen3-4B |
| Objective | Supervised fine-tuning (TRL SFTTrainer) |
| Dataset | Gemini-generated two-phase trajectories (HotpotQA, 6k) |
| Number of GPUs | 1 |
| Per-device training batch size | 8 |
| Gradient accumulation steps | 6 |
| Effective training batch size | 48 |
| Learning rate | 5e-5 |
| LR scheduler | Cosine |
| Warmup ratio | 0.1 |
| Weight decay | 0.01 |
| Number of epochs | 3 |
| Max sequence length | 1024 |
| Runtime precision | BF16 |
| Two-phase supervision | Implicit in the SFT trajectories |
| Special DB-lookup tokens | Enabled |
| Per-device evaluation batch size | 8 |
| Evaluation strategy | Epoch |
| Logging steps | 10 |
| Save strategy | Every 12.5% of an epoch |
| Maximum saved checkpoints | 8 |
Intended use
This checkpoint is intended for running and evaluating KBevo's two-phase knowledge construction and reasoning policy. It can be used with the official KBevo codebase to reproduce the reported results or as a starting point for further research and fine-tuning.
Citation
@inproceedings{Noonan2026:co-evolving,
title = {Co-Evolving Structured Knowledge and Reasoning in Language Models},
author = {Ryan Thomas Noonan and Linxi Zhao and Menghan Xu and Akanksha Sarkar and Mihir Mishra and Dongyoung Go and Kilian Q. Weinberger and Yoav Artzi and Jennifer J. Sun},
booktitle = {Proceedings of the Conference on Language Modeling},
year = {2026},
url = {https://arxiv.org/abs/2608.26386}
}