Qwen3.5-35B-A3B — Coding Extract (N128)
A training-free coding specialist extracted from Qwen/Qwen3.5-35B-A3B by analytically selecting which mixture-of-experts to keep — no fine-tuning, distillation, or gradient updates of any kind. Half the routed experts are removed; the always-on trunk (attention, shared expert, embeddings) is kept whole.
- Parameters: 18.54 B (parent: ~34.6 B) — 128 of 256 routed experts kept per layer
- Size: 37.1 GB (bf16,
model.safetensors) - Derived from: Qwen3.5-35B-A3B (hybrid linear-attention MoE, top-8 routing)
Usage
from transformers import AutoModelForCausalLM, AutoTokenizer
m = AutoModelForCausalLM.from_pretrained("deepgoyal19/qwen3.5-35b-a3b-code-extract-N128",
torch_dtype="bfloat16", device_map="auto")
tok = AutoTokenizer.from_pretrained("deepgoyal19/qwen3.5-35b-a3b-code-extract-N128")
Recommended coding config (from the parent's card): temperature=0.6, top_p=0.95, top_k=20, repetition_penalty=1.0, thinking mode on.
Evaluation
| Benchmark | pass@1 |
|---|---|
| HumanEval | 86.6% |
| HumanEval+ | 83.5% |
Provenance
census_export.json in this repo records the exact keep-set, N, selector, and parity metrics.