Qwen3.6-27B-A3B-CoderX
BF16 weights for CoderX — a long-horizon code prune of Qwen/Qwen3.6-35B-A3B: 256 experts per layer reduced to 184, ~35B → ~27B, still A3B active.
Same expert budget and the same competence map as the sibling Qwen3.6-27B-A3B-Coder — the whole difference is which 184 experts are kept. Our saliency ranking still drives the selection, but a REAP-style stability floor (p=24) overrides it per layer: the 24 experts we would drop that REAP ranks highest are rescued, paid for by evicting the 24 we rank lowest among our keeps. Exactly 24 experts per layer differ from the Coder cut, on all 40 layers.
The 72 evicted experts per layer are discarded, not merged. Every retained expert is
bit-identical to the base model's, and the build asserts it rather than assuming it
(verify_arm_identity.py → ARM_IDENTITY_OK, 0/40 layers mismatched). Router, attention and
norms are untouched. No fine-tuning, no distillation, no expert folding.
Built with omnimergekit.
Read this before you load it
- Routing is top-8 (
num_experts_per_tok: 8) — the base model's native setting, and measured rather than assumed: MBPP-full 0.784 / 0.790 at top-8 against 0.732 / 0.730 at top-10. The Coder sibling bakes top-10; this one does not. - The MTP block is included (
mtp.*, inmtp.safetensors, indexed) for speculative decoding. - This checkpoint is text-only.
architectures: Qwen3_5MoeForCausalLM,model_type: qwen3_5_moe_text— the base model's vision tower is not carried by this checkpoint — the merge-engine save path that produced it emits trunk modules only. The Coder sibling's safetensors repo is multimodal (Qwen3_5MoeForConditionalGeneration); this one is not. Thevision-<tier>tags on ollama get their vision tower from the Coder mmproj at the GGUF layer, so they are unaffected — but if you need a multimodal safetensors checkpoint, use Coder.
Quantised builds
- GGUF (19 tiers, all imatrix, MTP included):
Qwen3.6-27B-A3B-CoderX-MTP-GGUF - ollama:
ollama run mannix/qwen3.6-27b-a3b-coderx
Evaluation
Q6_K + imatrix, llama.cpp b9700, greedy (temperature 0.0 / top_p 1.0 / top_k 0), one
pinned serving geometry per bench, read back from the server log.
| Benchmark | CoderX | Coder (184e) | Qwen3.6-35B-A3B (256e) |
|---|---|---|---|
| LiveCodeBench v6 (77q, 24k think / 48k total) | 0.727 | 0.610 | 0.610 |
| HumanEval+ (164) | 0.970 | 0.951 | 0.939 |
| MultiPL-E-100 (rs+java+js, 300 completions) | 0.887 | 0.890 | 0.910 |
A same-basis repeat of MultiPL-E moved 1.0 pp on batch-scheduling nondeterminism alone, so the 0.33 pp CoderX↔Coder gap is a tie; the 2.33 pp gap to the teacher is real. Per language (CoderX / Coder / 256e): Rust 0.85 / 0.81 / 0.84 · Java 0.89 / 0.90 / 0.93 · JS 0.92 / 0.96 / 0.96.
The full canonical 9-bench suite has not been run on this checkpoint yet; non-code axes (GPQA, MATH-500, IFEval, ARC) are deliberately not quoted here.
Apache-2.0 · research checkpoint.
Tool-calling benchmark — tool-eval-bench hardmode (88 scenarios, 176 pts)
A3B-CoderX scores 137.4 ±4.9 of 176, seventh of ten — and +14.2 pts over A3B-Coder (123.2), the largest gap between any two models in this cohort's lower half. The CIs do not overlap.
The gain is concentrated exactly where Coder gives way: Hard Mode 24.6/38 (64.7%) vs 16.8/38 (44.2%), Autonomous Planning 3.8/6 (63.3%) vs 2.4/6 (40.0%), and Instruction Following 9.6/10 (96.0%) vs 8.0/10. Perfect on Tool Selection and Parameter Precision; Localization 5.6/6 (93.3%), Toolset Scale 7.6/8 (95.0%).
Weakest at Safety & Boundaries 17.6/26 (67.7%) and Context & State 15.0/20 (75.0%).
17 safety-critical failures across five seeds — TC-31, TC-34 and TC-60 on every seed,
plus TC-35 (Contradictory Parameters) on two. That is the highest count in the cohort;
if untrusted tool output is in scope for your deployment, read the safety notes below.
CoderX hits the TC-62 scorer crash on 3 of 5 seeds, so this figure is mildly depressed relative to a corrected harness.
Full cohort
| model | quant | Total Points (mean, 5 seeds) | 95% CI | safety-critical (5 seeds) |
|---|---|---|---|---|
| Qwen3.8-27B-Omnimerge-v6 | Q4_K_M | 156.4 ±3.5 | [152.0, 160.8] | 3 |
| Qwen3.8-27B (base) | UD-Q4_K_M | 150.8 ±2.5 | [147.7, 153.9] | 9 |
| Ornith-1.5-35B | IQ4_XS | 146.2 ±2.6 | [143.0, 149.4] | 10 |
| Qwen3.6-27B-Omnimerge-v4 | Q4_K_M | 146.2 ±2.7 | [142.9, 149.5] | 16 |
| Qwen3.6-27B (base) | Q4_K_M | 144.0 ±3.4 | [139.8, 148.2] | 14 |
| Qwen3.6-35B-A3B (base) | IQ4_XS | 141.6 ±2.4 | [138.6, 144.6] | 15 |
| Qwen3.6-27B-A3B-CoderX | Q4_K_M | 137.4 ±4.9 | [131.3, 143.5] | 17 |
| Ornith-1.5-27B-A3B-Coder | IQ4_XS | 136.8 ±4.8 | [130.9, 142.7] | 12 |
| Ornith-1.5-27B-A3B-CoderX | IQ4_XS | 134.0 ±2.5 * | [130.8, 137.2] | 14 |
| Qwen3.6-27B-A3B-Coder | Q4_K_M | 123.2 ±2.3 | [120.4, 126.0] | 15 |
* one seed (s42) is graded on 174 pts, not 176 — see that model's card.
- Scorer:
tool-eval-benchv2.6.0 (the pip/uv-installed package, verified viatool_eval_bench.__file__, not a git checkout). An earlier note in the runner claimedcf54b4b(v2.6.0-45); that is wrong and has been corrected — no cell ever ran it. All 50 cells ran the same v2.6.0, so the cohort is internally consistent. - v2.6.0 carries a known scorer crash on TC-62.
email_calls[-1]raisesIndexErrorwhen a model sent no valid CFO email; the orchestrator catches it and returns FAIL / 0 points while keeping the scenario in the denominator. It hits 11 of 38 scored cells, 2 pts each, and it is not neutral — it concentrates on the weakest models. Later harness commits credit that behaviour instead, so a fixed scorer would raise affected scores, unevenly. - 5 paired seeds [42–46], 64k context, context-pressure 0.25, max 8 turns, 120 s timeout, thinking enabled, sampler temp 0.6 / top-p 0.95 / top-k 20 (not greedy).
- Served on
llama.cpp b1788384120-c588c4f47with MTP speculative decoding enabled (nextn=YES spec=mtp), one model per GPU, sequential. - Quant tiers are not uniform across the cohort (Q4_K_M for the Omnimerge/A3B rows, IQ4_XS for Ornith and 35B-A3B, UD-Q4_K_M for the Qwen3.8 base). Cross-row gaps therefore carry a quantisation component and are not purely architectural.
- Do not pool these with the r/LocalLLaMA published tool-eval-bench figures: those were run at 256k context and are a different basis despite the shared scorer version.