KBevo-Qwen3-4B-GRPO

Final KBevo GRPO checkpoint (Qwen3-4B, step 500). Accompanies Co-Evolving Structured Knowledge and Reasoning in Language Models, published at the Conference on Language Modeling (COLM 2026).

KBevo: Structured knowledge + reasoning in two phases

KBevo teaches a single language model to answer multi-hop questions by first building a structured knowledge base and then reasoning over it:

1. Knowledge-base construction. Given a set of passages, the model extracts a compact structured KB of typed (entity, relation, value) triples. Inverse relations are added so the KB can be traversed in either direction.
2. Question answering through structured KB retrieval. For each question, the model issues typed KB lookups instead of re-reading raw text. Answers are grounded in retrieved triples rather than in parametric memory alone.

The two phases share weights and are trained jointly. SFT teaches the base extraction / lookup behaviour on synthesised two-phase rollouts. GRPO further optimises the joint policy on end-to-end HotpotQA answer-F1 rewards.

This repository

This repository contains the final KBevo GRPO checkpoint for Qwen3-4B (step 500). It is the checkpoint used for the paper's evaluation numbers below and is the recommended model for inference / evaluation.

Its SFT starting point is available at kilian-group/KBevo-Qwen3-4B-SFT.

All KBevo checkpoints

Full code, training, and evaluation scripts: https://github.com/kilian-group/KBevo

Loading

from transformers import AutoModelForCausalLM, AutoTokenizer

repo_id = "kilian-group/KBevo-Qwen3-4B-GRPO"
tokenizer = AutoTokenizer.from_pretrained(repo_id)
model = AutoModelForCausalLM.from_pretrained(repo_id, dtype="bfloat16")

For the two-phase KBevo inference agent (KB construction → structured QA), please use the reference implementation from the KBevo GitHub repository. See scripts/eval_kbevo.sh.

Results

Exact-match on the four benchmarks reported in the paper:

Training

Stage: Two-phase GRPO on HotpotQA, initialised from kilian-group/KBevo-Qwen3-4B-SFT (SFT checkpoint at step 735).

Hyperparameters:

Intended use

This checkpoint is intended for running and evaluating KBevo's two-phase knowledge construction and reasoning policy. It can be used with the official KBevo codebase to reproduce the reported results or as a starting point for further research and fine-tuning.

Citation

@inproceedings{Noonan2026:co-evolving,
  title         = {Co-Evolving Structured Knowledge and Reasoning in Language Models},
  author        = {Ryan Thomas Noonan and Linxi Zhao and Menghan Xu and Akanksha Sarkar and Mihir Mishra and Dongyoung Go and Kilian Q. Weinberger and Yoav Artzi and Jennifer J. Sun},
  booktitle     = {Proceedings of the Conference on Language Modeling},
  year          = {2026},
  url           = {https://arxiv.org/abs/2608.26386}
}