Qwen3.5-35B-A3B-Asis-GRPO
Asis-GRPO is a reinforcement-learning continuation of Qwen3.5-35B-A3B-Asis (a web-search-agent SFT of Qwen/Qwen3.5-35B-A3B) trained with turn-level GRPO on stop-decision pivot states (PivotRL-style). Instead of optimizing whole-episode returns, the policy is optimized specifically at the decision points where an agent chooses between stopping to answer and continuing to search — the locus of over-reflection. Qualitatively, the released checkpoint maintains baseline accuracy while substantially reducing post-answer over-search.
This repository contains the full serving weights (13 safetensors shards, ~65 GB, bf16) plus tokenizer, config, and chat template.
Model description
- Architecture: Qwen3.5-35B-A3B — a mixture-of-experts (MoE) decoder-only transformer with ~35B total parameters and ~3B active per token. Architecture, tokenizer, and chat/tool-calling template are unchanged from the base model.
- Lineage: Qwen3.5-35B-A3B → SFT on an in-house web-search agent trajectory corpus, used unmodified (Asis arm, 131,072-token training context) → turn-level GRPO (this model).
- Behavior domain: multi-turn web-browsing agentic tool use (search / page-open / in-page find style tool surface) with explicit reasoning.
Training procedure (RL stage)
Turn-level GRPO on stop-decision pivot states: mid-trajectory states where the agent has just produced (or is about to produce) enough evidence to answer, and must decide whether to stop or keep searching. Rollouts are generated from each pivot state and rewarded by a deterministic stop-verifier — a rule-based checker of whether the continuation stops appropriately and remains grounded — rather than by a learned reward model.
| Component | Value |
|---|---|
| Algorithm | GRPO (group-relative policy optimization), turn-level |
| Group size | K = 8 rollouts per pivot state |
| Reward | deterministic stop-verifier (rule-based; no learned reward model) |
| KL regularization | to the frozen SFT (Asis) reference policy, coefficient 0.01, low-variance KL estimator |
| Global batch size | 128 sequences (rollout batches of 16 states x K = 8) |
| Released checkpoint | ~32 effective RL steps, selected by a downstream step-sweep |
Checkpoint selection
The RL run was intentionally trained past its useful region and swept checkpoint-by-checkpoint on downstream browsing-agent evaluation. The released step (~32) is the accuracy-neutral over-search-reduction point: the latest checkpoint at which end-task accuracy is statistically indistinguishable from the SFT baseline while post-answer over-search (continued searching after the answer has been stated) is significantly reduced. Later checkpoints trade accuracy or healthy verification behavior for further efficiency and were not released.
Intended use
Research artifact. Intended for studying stop-decision RL, over-search reduction, and the accuracy-efficiency frontier of tool-using web agents; also usable as a more search-efficient drop-in for the Asis SFT arm on the same tool surface (function-calling tools for web search, page opening, and in-page lookup).
Limitations
- Research artifact, not a production assistant; behavior is tuned for a specific browsing-agent tool surface and prompt format.
- The safe RL window is base-dependent. The ~32-step selection is specific to this SFT initialization; on other initializations the beneficial window opens and closes at different points (see Qwen3.5-35B-A3B-Correct-GRPO, whose sweet spot is much earlier). Do not assume the step choice, or the checkpoint itself, transfers across bases.
- The stop-verifier is a rule-based proxy for good stopping behavior; the model may have partially adapted to the verifier's notion of evidence access, so efficiency gains should be re-validated under any new tool surface or reward definition.
- Only a small number of RL steps were applied; capabilities outside the stop-decision behavior are essentially those of the SFT parent.
- No additional safety alignment was applied beyond what the base model provides.