S3 Qwen3 4B starts from
Qwen/Qwen3-4B-Thinking-2507
and is trained with on-policy reinforcement learning in the S3 local search
environment.
At a glance
| Property | Value |
|---|---|
| Base model | Qwen3-4B-Thinking-2507 |
| Checkpoint | Global step 120 |
| Training | verl GRPO with SEAL |
| Training questions | ~1,613 value-answer items in difficulty band [1, 7] |
| Environment | Nine tools over ~1.12M computer-science papers |
Evaluation
The checkpoint was evaluated with the native AstaBench tool surface on the
full-text-available LitQA2 test subset (n=75).
| Model | Accuracy |
|---|---|
| Qwen3-4B-Thinking-2507 base | 0.173 |
| S3 Qwen3 4B, step 120 | 0.453 |
Accuracy increased through step 120 and declined at later checkpoints. Transfer to the AstaBench paper-finder task remained approximately flat, suggesting that the gain is specific to QA-like tasks.
Step 120 was selected after inspecting this test-set checkpoint curve. The reported result is diagnostic rather than an unbiased final benchmark.
Load the model
from transformers import AutoModelForCausalLM, AutoTokenizer
model_id = "trillionlabs/sim-scholar-qwen3-4b"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
model_id,
device_map="auto",
torch_dtype="auto",
)
Reproducing the reported behavior requires compatible tool definitions, prompting, observation formatting, and stopping behavior. These components are available in the linked code repository.
Intended use
This checkpoint is intended for research on literature-search agents, tool-using language models, agentic reinforcement learning, and sim-to-real transfer.
Limitations
- The model can hallucinate papers, citations, or answers.
- Reported performance comes from a small, post hoc selected evaluation.
- Training used synthetic questions over a computer-science-only corpus.
- Behavior depends strongly on the tool surface and retrieval backend.
- Value-answer training did not improve paper-finding performance.
The checkpoint should not be used as an authoritative source of scientific or medical information.
License
This checkpoint is licensed under the Apache License 2.0, consistent with the base model. Training-data and third-party material retain any applicable original terms.