Qwen3.6-35B-A3B — QD (Quadux Dynamic) GGUF

QD = Quadux Dynamic Quantization: per-tensor quantization maps built from measured sensitivity — not from a fixed rule ladder. Every byte of a given size budget is placed where measurements say it buys the most quality.

All files include the MTP head (blk.40, verified per file) and work with the included vision projector (mmproj-F16.gguf). Base model: Qwen/Qwen3.6-35B-A3B — MoE, 35.5B total / ~3B active, hybrid attention (256k context at very small KV cost).

The files

Seven rungs, each a measured QD allocation. KLD = KL divergence vs the BF16 model (lower = closer to full precision), measured on wikitext-2 (en) and a German Wikipedia sample (de). Δ is versus unsloth's build at the same byte size. Full data — including the rungs where unsloth wins — is in BENCHMARKS.md.

We measured nine rungs and publish these seven; at Q2_K_XL, Q5_K_XL and Q6_K_XL unsloth's build is better and we link to theirs instead of shipping a worse file.

QD-Q5_K note: at 26.42 GB it is our largest rung, chosen to fit a 32 GB card with real context (see the fit table below). unsloth's UD-Q5_K_XL (27.16 GB) is higher quality per byte (0.00774 vs 0.00906) but does not fit a 32 GB card once you add context — take theirs if you have the room, ours if you need the context.

Where unsloth wins — use theirs, not ours:

Why: our sensitivity calibration is measured around the Q3–Q4 working point. Near Q5/Q6 the remaining damage is so small (KLD < 0.008) that our probe resolution no longer resolves it, and at Q2 the type ladder we are willing to ship bottoms out. We could have shipped those files anyway. We didn't.

We then measured why, with a gradient sweep at the Q6 point: the marginal value ratio of the expert classes there is down : gate : up = 9.4 : 1.6 : 1 — exactly the allocation unsloth's high rungs use. Our own numbers reproduce their rule; our optimizer's error proxy does not find it. A rule-based rebuild using that ratio came out worse (0.0116 vs 0.0091 at the 32G budget), because single-step gradients do not extrapolate to multi-step jumps. Both experiments are documented in BENCHMARKS.md §6. Above Q5, unsloth's maps are at or near the optimum of the type ladder, and we say so.

Method

unsloth's Dynamic GGUFs pioneered per-tensor type selection; their exact selection algorithm is proprietary. QD is an independent, fully measured re-implementation of the idea (not of their algorithm):

1. Build uniform reference quants (Q2_K…Q8_0, IQ2…IQ4) of the full model.
2. Compute an imatrix-weighted, norm-relative error matrix per tensor and candidate type — with per-expert activation statistics for MoE tensors.
3. Calibrate that proxy against measured class probes: force one tensor class to Q2_K at a time, measure true KL divergence against BF16 logits.
4. Knapsack-optimize the per-tensor type map to an exact byte budget.
5. Verify with a measured gradient sweep (each class ±1 type step) and keep only profitable moves.

We gratefully use unsloth's published imatrix for this model (Apache-2.0) — credit to them; the type maps, measurements and pipeline are our own. Tensors the pipeline cannot judge — the MTP block blk.40, which standard imatrix calibration never activates — copy the corresponding unsloth choices from the UD file of the same size class (getting this wrong cost us 22% at Q3 until we found it; see BENCHMARKS.md).

The per-tensor maps are published in tensortypes/ so anyone can rebuild or audit any file.

Task behaviour (measured, not claimed)

Which file for your GPU — measured, not estimated

Pick by the VRAM you actually have free (run nvidia-smi first — your OS and browser already hold some). We ran the two extremes end-to-end on real cards; the rest follow the same rule.

⚠️ The desktop-display tax is real — we measured it. QD-Q4_K (19.5 GB)

• 256k + vision on a 24 GB laptop (RTX 5090, Windows): loaded at 98% VRAM with a bare desktop (~0.5 GB display) and failed to load at all with a browser open (~9 GB display → 19.5 GB weights no longer fit 15 GB free). Headless on a 32 GB card, QD-Q5_K fits 1×256k+vision but OOMs at 2×256k. Rule of thumb: usable = card size − what your OS already holds. Subtract ~1 GB headless, 2–9 GB on a desktop with a browser. When in doubt, pick the next file down or reduce context.

Naming note: HuggingFace's file widget shows each quant by its base type (Q4_K, Q2_K, …) and size only — that is why we name files by standard quant class and put the card-fit guidance here, in this table, rather than in the filename. Two Q4 rungs exist on purpose: QD-Q4_K_XL (22.9 GB, more quality) and QD-Q4_K (19.5 GB, fits 24 GB).

Smallest rung: with IQ2_XXS as the lowest type we ship, QD-IQ2_XXS at ~10.6 GB is the floor; below that we'd need sub-IQ2 types we consider unusable for this model.

8 GB cards: don't chase a ~1.35 bpw file. Use MoE offload — 88% of the bytes are expert tensors but only ~3B parameters are active per token:

llama-server -m Qwen3.6-35B-A3B-QD-Q4_K.gguf --n-cpu-moe 41 \
  --ctx-size 32768 -ngl 999 --flash-attn on --jinja

Experts stay in system RAM, attention and KV on the GPU — near-full quality on a small card, at the cost of throughput.

If you want Q8 or higher

At Q8 the damage is essentially gone (0.0056 KLD versus 0.012 at Q4), so a "QD-Q8" would be a label on a file practically identical to any standard Q8_0. We link instead of duplicating:

Note: Qwen does not publish GGUF — the Q8_0 and BF16 GGUFs come from unsloth (Apache-2.0), same source as the imatrix we build on.

Serving (llama.cpp)

llama-server -m Qwen3.6-35B-A3B-QD-Q4_K_XL.gguf \
  --jinja --flash-attn on -ngl 999 \
  --mmproj mmproj-F16.gguf \
  --ctx-size 262144 --parallel 4 \
  --spec-type draft-mtp,ngram-mod --spec-draft-n-max 1 \
  --temp 0.6 --top-p 0.95 --top-k 20

The warning unused tensor blk.40… ignoring is cosmetic (llama.cpp #26765) — MTP is active when the log shows creating MTP draft context. For batch classification pass "chat_template_kwargs": {"enable_thinking": false} per request.

Honesty notes

• KLD is measured against BF16 logits of this model, 60 chunks × 512 tokens per language. It is a sharp metric for quantization damage, not a task suite.
• Type maps were optimized on the mean of English and German KLD. A German-calibrated QD-DE line is planned separately.
• Every head-to-head is at byte parity: the QD file is never larger than the unsloth file it is compared with.
• We publish the rungs where we measured an improvement, link to unsloth where they measured better, and say so in both directions.

Deutsche Kurzfassung

QD (Quadux Dynamic) verteilt ein Byte-Budget nach gemessener Per-Tensor- Empfindlichkeit (imatrix-gewichtete Fehlermatrix, Klassen-Sonden, Knapsack, Gradient-Verifikation) statt nach festen Regeln. Von neun gebauten Stufen veröffentlichen wir die sieben, in denen die Messung für uns spricht — bei Q2_K_XL, Q5_K_XL und Q6_K_XL ist unsloths Variante besser, und wir verlinken sie statt eine schwächere eigene Datei anzubieten.

Die Dateien tragen Standard-Quant-Namen (QD-Q4_K, QD-Q2_K …); welche Datei auf welche Karte mit welchem gemessenen Kontext passt, steht in der Tabelle „Which file for your GPU". Auf einer echten Desktop-Karte mit Monitor geht real 1–9 GB VRAM fürs Display ab (gemessen) — also weniger Kontext als headless. Auf Deutsch fällt der QD-Vorsprung durchweg größer aus als auf Englisch. Für 8-GB-Karten empfehlen wir MoE-Offload statt Extremquantisierung.

Quantisiert von Quadux IT. Basismodell © Alibaba Cloud (Apache-2.0). imatrix von unsloth (Apache-2.0); Methode, Typkarten und Messungen sind eigene Arbeit.

License

This model and its base model are licensed under the Apache License 2.0, which permits commercial and research use, modification, and redistribution, subject to the standard requirements: include the copyright notice, the license text, and a NOTICE of any changes.

• Base model: Qwen/Qwen3.6-35B-A3B — Apache 2.0, © Alibaba Cloud.
• This repository: Apache 2.0 (same terms as the base model).
• Modifications by Quadux IT GmbH: GGUF quantization only — QD (Quadux Dynamic) per-tensor type maps, built on unsloth's published imatrix (Apache 2.0) and MTP conversion. No change to the base weights or architecture.

If you redistribute these files, include the Apache 2.0 license text and attribution to the upstream Qwen team, to unsloth (imatrix + MTP conversion), and to Quadux IT GmbH. See NOTICE and LICENSE.

Citation

The original Qwen3 work — cite this if you publish results using the model:

@misc{qwen3.6,
  title  = {Qwen3.6-35B-A3B},
  author = {Qwen Team},
  year   = {2026},
  publisher = {Hugging Face},
  howpublished = {\url{https://huggingface.co/Qwen/Qwen3.6-35B-A3B}}
}

To cite this quantization specifically:

@misc{quadux_qd_qwen3_6_35b,
  author       = {{Quadux IT GmbH}},
  title        = {Qwen3.6-35B-A3B-QD-GGUF (Quadux Dynamic Quantization)},
  year         = {2026},
  publisher    = {Hugging Face},
  howpublished = {\url{https://huggingface.co/QuaduxIT/Qwen3.6-35B-A3B-QD-GGUF}},
  note         = {Measured per-tensor GGUF quantization of Qwen/Qwen3.6-35B-A3B; built on unsloth's imatrix and MTP conversion}
}

About Quadux IT GmbH

Software for engineering offices and accounting pipelines. Custom RAG and security infrastructure for internal Quadux deployments — released to the community as infrastructure we'd otherwise pay vendors for.

Find more at quadux.it · contact info@quadux.it

Imprint

Quadux IT GmbH · Schulstr. 3 · 37139 Adelebsen · Germany Registered Göttingen, HRB 206773 · VAT ID DE353975332 · DUNS 344198559 Managing Director: Walter Hoffmann