Qwen3.5-35B-A3B-Correct-GRPO
Correct-GRPO is a reinforcement-learning continuation of Qwen3.5-35B-A3B-Correct (the correctness-filtered SFT arm of a four-arm web-search-agent study on Qwen/Qwen3.5-35B-A3B) trained with turn-level GRPO on stop-decision pivot states (PivotRL-style). It applies the same RL recipe as Qwen3.5-35B-A3B-Asis-GRPO to a different SFT initialization, and exists primarily as the study's evidence that the safe RL window is base-dependent: on this initialization the beneficial region closes far earlier, and the released checkpoint sits at that earlier sweet spot (~14 effective RL steps vs ~32 for the Asis base).
This repository contains the full serving weights (13 safetensors shards, ~65 GB, bf16) plus tokenizer, config, and chat template.
Model description
- Architecture: Qwen3.5-35B-A3B — a mixture-of-experts (MoE) decoder-only transformer with ~35B total parameters and ~3B active per token. Architecture, tokenizer, and chat/tool-calling template are unchanged from the base model.
- Lineage: Qwen3.5-35B-A3B → SFT on the correctness-filtered subset of an in-house web-search agent trajectory corpus (deterministic answer matching; Correct arm, 131,072-token training context) → turn-level GRPO (this model).
- Behavior domain: multi-turn web-browsing agentic tool use (search / page-open / in-page find style tool surface) with explicit reasoning.
Training procedure (RL stage)
Turn-level GRPO on stop-decision pivot states: mid-trajectory states where the agent must decide between stopping to answer and continuing to search. Rollouts from each pivot state are rewarded by a deterministic stop-verifier (rule-based; no learned reward model).
| Component | Value |
|---|---|
| Algorithm | GRPO (group-relative policy optimization), turn-level |
| Group size | K = 8 rollouts per pivot state |
| Reward | deterministic stop-verifier (rule-based; no learned reward model) |
| KL regularization | to the frozen SFT (Correct) reference policy, coefficient 0.01, low-variance KL estimator |
| Global batch size | 128 sequences (rollout batches of 16 states x K = 8) |
| Released checkpoint | ~14 effective RL steps, selected by a downstream step-sweep |
Checkpoint selection and the base-dependent window
As with the Asis-based run, the RL run was swept checkpoint-by-checkpoint on downstream browsing-agent evaluation, and the released step is the accuracy-neutral over-search-reduction point for this base. The step-sweep found that on the correctness-filtered initialization, accuracy regression sets in several times earlier and more sharply than on the unfiltered Asis initialization — hence the much earlier selected checkpoint. This asymmetry is a central observation of the study: the same stop-decision RL recipe has a base-dependent safe window, and checkpoints (or step budgets) must not be transplanted across initializations.
Intended use
Research artifact. Intended for studying how the SFT initialization shapes the safe window of stop-decision RL, and for arm-invariance / generality analyses alongside its Asis-based counterpart. It expects a browsing-agent tool surface compatible with its training format (function-calling tools for web search, page opening, and in-page lookup).
Limitations
- Research artifact, not a production assistant; behavior is tuned for a specific browsing-agent tool surface and prompt format.
- Narrow safe window. This base tolerates very little stop-decision RL before regressing; the released checkpoint is close to that edge. Any continued training from this checkpoint should re-run the step-sweep rather than reuse the Asis-base schedule.
- The stop-verifier is a rule-based proxy for good stopping behavior; efficiency characteristics should be re-validated under any new tool surface or reward definition.
- Only a small number of RL steps were applied; capabilities outside the stop-decision behavior are essentially those of the SFT parent.
- No additional safety alignment was applied beyond what the base model provides.