Qwen3.5-35B-A3B-Drop
Drop is the typed-deletion arm of a four-arm controlled study of training-data interventions against over-reflection in web-search agents. It is a supervised fine-tune of Qwen/Qwen3.5-35B-A3B on the same in-house web-search agent trajectory corpus as the Asis baseline, except that whole trajectories exhibiting pathological over-reflection types are deleted before training. Deletion is typed: trajectories are labeled against an A–G taxonomy of over-reflection behaviors, and the pathological types are removed in their entirety (no editing of surviving trajectories).
This repository contains the full serving weights (13 safetensors shards, ~65 GB, bf16) plus tokenizer, config, and chat template.
Model description
- Architecture: Qwen3.5-35B-A3B — a mixture-of-experts (MoE) decoder-only transformer with ~35B total parameters and ~3B active per token. Architecture, tokenizer, and chat/tool-calling template are unchanged from the base model.
- Training context length: 131,072 tokens (full agent trajectories, multi-turn with interleaved tool calls and tool results).
- Behavior domain: multi-turn web-browsing agentic tool use (search / page-open / in-page find style tool surface) with explicit reasoning.
Training procedure
Full-parameter supervised fine-tuning on a Megatron-based distributed training stack. Identical recipe to the sibling arms; only the data intervention differs.
| Hyperparameter | Value |
|---|---|
| Epochs | 2 |
| Global batch size | 128 |
| Learning rate | 5e-6, cosine decay to 5e-7 |
| Sequence length | 131,072 |
| Objective | token-level cross-entropy on assistant turns (trajectory imitation) |
Intervention: typed whole-trajectory deletion
The study's A–G taxonomy of over-reflection behaviors categorizes pathological patterns in browsing-agent trajectories — e.g., post-answer verification loops (the agent keeps searching after it has already stated the answer), redundant re-search after sufficient evidence has been gathered, and ungrounded reasoning not supported by retrieved content. In this arm, any trajectory carrying a pathological type is removed entirely. This is the coarsest intervention class in the study: it trades corpus size for corpus cleanliness, in contrast to the surgical editing of the Repair arm and the outcome-based filtering of the Correct arm.
| Arm | Intervention class |
|---|---|
| Asis | none — unmodified trajectory imitation baseline |
| Drop (this model) | typed whole-trajectory deletion of pathological over-reflection types |
| Repair | taxonomy-guided surgical repair (truncate post-evidence over-search tails; remove ungrounded trajectories) |
| Correct | correctness-filtered subset (deterministic answer matching) |
Intended use
Research artifact. Intended for studying how removing pathological demonstrations (as opposed to editing them, or filtering by outcome) affects over-reflection, stopping behavior, and search efficiency in tool-using web agents. It expects a browsing-agent tool surface compatible with its training format (function-calling tools for web search, page opening, and in-page lookup).
Limitations
- Behavior is tuned for the browsing-agent tool surface described above; it is not a general-purpose assistant fine-tune.
- Whole-trajectory deletion shrinks the effective training set relative to the Asis baseline; capability differences between arms confound data quality with data quantity by construction (this is part of what the four-arm design measures).
- No additional safety alignment was applied beyond what the base model provides.
- Trained primarily for English-language reasoning traces.