Qwen3.6-11B-A3B-Niwaki-4bit-mlx
Qwen3.6-35B-A3B pruned to 11B total / 3.05B active
parameters, stored at 4-bit — 7.6× less expert memory than
the 8-bit reference. Stock mlx_lm, no custom code.
Niwaki (庭木) are Japan's garden trees, sculpted by meticulous pruning so that every branch serves the form of the whole. This model applies that spirit to a Mixture-of-Experts: every routed expert is individually width-pruned using statistics of its routed tokens over a code-weighted mixed corpus (source from The Stack, code-reasoning traces, chat, and web), reconstructed to compensate, then distilled from the full model on the same mixed corpus, and stored at low precision. The result is an ordinary MoE — smaller, same architecture, standard fused kernels.
A paper with the full method and measurements is coming soon.
Benchmarks
Full evaluation protocol: WikiText-2 perplexity over 256 × 2048-token windows; task average over the complete arc_easy, hellaswag, piqa, winogrande, and boolq test sets, paired against the reference.
| model | expert storage | wt2 ppl ↓ | task avg ↑ | retention | generation d2 (avg/min) |
|---|---|---|---|---|---|
| reference (8-bit) | 34.2 GB | 6.76 | 0.796 | 100% | 0.89 / 0.77 |
| this model | 4.5 GB (0.132×) | 16.88 | 0.664 | 83.5% | 0.82 / 0.62 |
Generation quality is measured, not assumed: an 8-prompt battery (code, reasoning, chat, creative; 600-token sampled generations) scored by bigram diversity. Known limitation (measured): sustained code generation is the weakest axis of this model (battery minimum 0.62 on long-file code prompts, vs 0.75+ for its larger siblings); output tends to truncate early rather than stay coherent to the end of a long file. Chat, reasoning, and short-form writing are functional. Choose the 19B for heavy code work.
Model dimensions
| total parameters | 11B (from 35B) |
| active parameters per token | ~3.05B (from ~3.8B: 2.8B backbone + 8 pruned experts) |
| layers / routed experts / top-k | 40 / 256 / 8 |
| expert intermediate size | 128 (from 512) |
| expert precision | 4-bit (group 64) |
| backbone precision | 8-bit (untouched) |
| whole-model size on disk | ~7.4 GB |
| runs comfortably in | 12 GB unified memory |
Usage (MLX, Apple Silicon)
from mlx_lm import load, generate
model, tokenizer = load("neopolita/Qwen3.6-11B-A3B-Niwaki-4bit-mlx")
print(generate(model, tokenizer, prompt="...", max_tokens=256))
GGUF builds for llama.cpp: Qwen3.6-11B-A3B-Niwaki-4bit-GGUF (UD-Q3K recommended + Q4_K_M).
Notes and limitations
- Compression trades quality: this model sits at ~84% of the reference task average and 2.50× its perplexity. Choose the family member that fits your memory budget.
- Evaluated text-only on English-web-heavy data; the base model's biases are inherited and rare-domain behaviour of pruned experts is less tested.
- Part of the Niwaki family: 27B-A3B-Niwaki-2bit · 19B-A3B-Niwaki-2bit.
Base model by the Qwen team (Apache 2.0); 8-bit MLX conversion by mlx-community; pruning and distillation by the Niwaki project, 2026-08. Method details: paper coming soon.