Qwen3.5-4B · Video-Evaluation GRPO
This directory contains the Hugging Face conversion of the final checkpoint
from the video-evaluation GRPO run initialized from
JianhuiWei/qwen35_4b_sft_VE.
Training summary
- Initial model:
JianhuiWei/qwen35_4b_sft_VE - RL algorithm: GRPO
- Agent harness: OpenCode
- Final checkpoint: global step 294
- Samples per prompt: 16
- Learning rate:
1e-6with a constant schedule - KL regularization: low-variance KL with coefficient
0.001 - Advantage normalization: enabled
- Error Tool-Call Mask in the initial SFT model: disabled
The full effective configuration, artifact provenance, and Step-294 evaluation snapshot are recorded in RL_TRAINING_METADATA.md.
Custom chat template
The bundled chat_template.jinja is the custom Qwen3.5 template used by the
training and inference stack. It preserves assistant reasoning content in
<think> blocks and renders tool calls in Qwen's XML-style tool-call format.
When serving with SGLang, pass the template explicitly:
python -m sglang.launch_server \
--model-path /path/to/model \
--chat-template /path/to/model/chat_template.jinja \
--reasoning-parser qwen3 \
--tool-call-parser qwen3_coder
Transformers loading
import torch
from transformers import AutoProcessor, Qwen3_5ForConditionalGeneration
model_id = "/path/to/model"
processor = AutoProcessor.from_pretrained(model_id)
model = Qwen3_5ForConditionalGeneration.from_pretrained(
model_id,
torch_dtype=torch.bfloat16,
device_map="auto",
)
The checkpoint is stored as sharded Hugging Face safetensors. The language weights come from RL step 294; the frozen visual encoder is preserved from the initial SFT model. Optimizer, scheduler, trainer, and RNG states are intentionally excluded.