Qwen3.8-27B-MatRL-MT-RL
Multi-turn RL for novel inorganic crystal structure generation, trained with tool use against a machine-learned interatomic potential (MLIP) reward.
This is the 27B arm of the MatRL scale series. The 4B and 9B arms are Qwen3.5-4B-MatRL-MT-RL and Qwen3.5-9B-MatRL-MT-RL.
Intermediate checkpoint. This is step 100 of RL stage 2c — roughly 170 of a planned 400 cumulative RL steps. It is released as a work-in-progress snapshot, not a final model. See Known issues before using it.
Base model
Built on Qwen/Qwen3.8-27B (27.36B params, 64 layers, hidden size 5120, hybrid
linear/full attention).
Note that config.json reports model_type: "qwen3_5" and
Qwen3_5ForConditionalGeneration — Qwen3.8 is a version bump of the Qwen3.5 architecture,
not a new one. The tokenizer and chat template are Qwen3.8's and differ from Qwen3.5's;
use the ones shipped in this repo rather than substituting Qwen3.5's.
Training pipeline
| Stage | Steps | Note |
|---|---|---|
| Cold-start SFT | 420 | multi-turn corpus, filtered at seq_len 16384 through this model's renderer |
| RL stage 1 | 30 | |
| RL stage 2 | 10 | |
| RL stage 2b | 30 | |
| RL stage 2c | 100 | this checkpoint |
Cumulative: 170 RL steps on top of the SFT.
Task and reward
The model works in a multi-turn tool loop (up to 10 turns) proposing crystal structures, then submits a final candidate. Task mix: unconditional generation, target formula, target element set, and property-conditioned targets (band gap, bulk modulus).
Tools: retrieve_materials, propose_structure, substitute_element, check_novelty,
evaluate_structure, predict_band_gap, calculate_bulk_modulus, submit.
Reward is a weighted rubric over submission, validity, novelty, honest stability (MLIP
relaxation, equflashv2 backend), SUN bonuses, property-target match, group diversity,
efficiency, and format.
Training setup
- Framework: prime-rl 0.7.0, GRPO-style with group-relative advantages
- Hardware: 8×H200 (5 trainer / 2 inference / 1 MLIP reward pool)
- Batch: 128 rollouts, group size 16, oversampling 1.25
- Sampling: temperature 1.15,
max_completion_tokens2048,max_model_len32768 - Optimizer: fp32 optimizer state, constant LR, FSDP with CPU offload
Known issues
- Intermediate, not converged. Mean reward is roughly flat across stage 2c (~3.2 → ~3.4), within run-to-run noise.
- Tool-call batching drift. Over stage 2c the policy progressively packs more tool calls into a single assistant message (~3 at step 1 → ~7 by step 100, with batches up to 27). Around 41% of rollouts at this checkpoint hit the 2048-token completion cap mid-batch, which can emit a truncated final tool call. Downstream users should expect occasional malformed trailing tool calls and may want a larger completion budget or a cap on tool calls per message.
- The stage-2c run later terminated at step 122 (vLLM engine timeout on very long prompts, a downstream consequence of the same drift). This checkpoint predates that failure.
Usage
from transformers import AutoModelForCausalLM, AutoTokenizer
model_id = "amayuelas/Qwen3.8-27B-MatRL-MT-RL"
tok = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, dtype="auto", device_map="auto")
The model expects the multi-turn tool-calling format it was trained in; use the bundled
chat_template.jinja.
License
Inherits the license of the base model, Qwen/Qwen3.8-27B.