KAT-Coder-V2.5-Dev — REAP-205E — MTP — GGUF

Kwaipilot/KAT-Coder-V2.5-Dev with its MoE experts pruned from 256 to 205 per layer (19.92%), MTP head grafted, published as a bf16 master plus imatrix GGUF tiers.

Pruned with REAP (Router-weighted Expert Activation Pruning, arXiv 2510.13999), calibrated on 2.6M tokens of KAT's own generated output — not on someone else's dataset.

~34.7B → ~28.3B parameters. Same 8 active experts per token, same architecture, same 262k vocab, same context.

Measured quality — read this before downloading

KL-divergence of this model against the unpruned BF16 original, both at bf16, same text, 39 chunks at n_ctx 4096:

How to read it. The median KLD of 0.0015 says half of all tokens are essentially untouched. The mean of 0.0588 is pulled up by a minority of tokens that diverge sharply (max KLD 14.56). Same-top-1 of 94.6% means roughly 1 token in 18 picks a different most-likely candidate than the original.

In familiar units: this is approximately the quality cost of a good 4-bit quantization — mean KLD 0.059 sits in the usual Q4_K_M band. It is not lossless, and this card will not claim it is.

Two honest caveats:

1. The KLD text is drawn from the same corpus family used to compute the saliency. Experts were selected to matter for that distribution, so 94.6% is an optimistic bound — expect more divergence further from code/reasoning text.
2. No downstream coding benchmark (HumanEval/MBPP/SWE-bench) has been run. Nothing here should be read as a retention claim on those.

Why 205 experts and not some rounder number

The prune rate was chosen from the measured saliency distribution, not by analogy to other models. Measured on the real weights, per layer:

KAT's expert saliency is long-tailed but shallow: mean Gini 0.232, and the coldest quarter of experts still carries 13.6% of routed saliency where a uniform distribution would give 25%. The marginal cost of each removed expert rises smoothly from 0.45 to 0.89 of an average expert with no knee anywhere — there is no natural cut point in this model, so the rate is a judgement, and 205 sits at the top of the cheap region.

Skew is U-shaped in depth: the last blocks (38–39) and block 0 are markedly long-tailed and cheap to cut, while blocks 16–24 are nearly flat. Layer 20 is the worst layer to cut at every rate below 50%. A per-layer schedule would beat a uniform rate, but n_expert is a single scalar in GGUF, so a uniform count is what the format allows.

Routing is not language-specialised. Across 200k EN and 201k ZH tokens, zero experts fired in exactly one language and not the other, on all 40 layers — so an English-heavy calibration set does not endanger Chinese.

How it was calibrated

• 2,516 rows / 2,607,981 tokens, generated by KAT-Coder-V2.5-Dev itself at Q6_K on real prompts (code-instruct, repair, tool-use, reasoning, Chinese), 100% carrying a genuine <think> block. Real prompts, native completions — the model's own token distribution, not a third party's.
• Saliency S_j = (1/N_j) Σ g_j(t)·‖f_j(t)‖₂, accumulated per layer over a single forward pass, ranked independently per layer.
• Ranking by activation saliency, not by usage frequency: keep-set agreement between the two is only 0.627, so hit-count pruning would cut a substantially different set.

Convergence caveat: keep-set Jaccard between half the corpus and the full corpus is 0.977 against a 0.98 stopping bar — roughly one corpus doubling short. The binding constraint is prompt diversity (2,516 prompts), not token count. Experts near the cut line could shift slightly with more calibration.

The MTP head — an explicit approximation

KAT ships no mtp.* tensors; the multi-token-prediction head is grafted from Qwen/Qwen3.6-35B-A3B (19 tensors, verified byte-identical to the donor). Because transformers drops mtp.* on load, no activation saliency exists for the head's experts.

Since n_expert is one scalar for all blocks, the head had to be cut to 205 too. It was pruned by per-expert weight magnitude — data-free, computed from the actual tensors being cut. This is an approximation and may be revised.

Verify it yourself the cheap way: draft acceptance on the unpruned model measures 0.90–0.94 with --spec-type draft-mtp,ngram-mod. If the approximation were bad, acceptance would collapse.

Files

Same nine tiers, same per-tensor recipes as the unpruned release.

Tier ordering is readable from the recipes. UD-Q5_K_S dominates UD-Q4_K_XL: identical Q8_0 pins on output/embd/shared-expert/attention, and the only difference is routed gate/up experts held at Q5_K rather than a Q4_K/Q5_K mix. UD-Q4_K_XL is the size/quality pick, not the quality pick.

The imatrix could not be inherited. ffn_*_exps.weight is a fused tensor whose imatrix entry is dimensioned by expert count, so a 256-expert imatrix — including the standard APEX blend — cannot apply to a 205-expert model. That is a shape mismatch, not a preference. This one was computed from scratch on the pruned weights using the same KAT-native corpus that drove the pruning.

Every quant here is reproducible from the BF16 master; the master is reproducible from the original weights plus keep-index.json.

Usage

Load the master from its first shard; llama.cpp finds the rest.

llama-server -m Kwaipilot_KAT-Coder-V2.5-Dev-REAP-205E-MTP-UD-Q4_K_XL.gguf \
  -ngl 99 -c 32768 -fa on --jinja -ub 2048 -b 2048 \
  --cache-type-k q8_0 --cache-type-v q8_0 \
  --spec-type draft-mtp,ngram-mod \
  --spec-draft-n-max 1 --spec-draft-n-min 0 --spec-draft-p-min 0.75 \
  --spec-ngram-mod-n-min 8 --spec-ngram-mod-n-max 24 --spec-ngram-mod-n-match 48 \
  --reasoning-format deepseek

-ub 2048 roughly doubles generation throughput over the 512 default. With -np N -c C each slot gets C/N tokens — overrun it and the server drops connections silently.

Provenance

• Base: Kwaipilot/KAT-Coder-V2.5-Dev
• MTP donor: Qwen/Qwen3.6-35B-A3B
• Method: REAP, arXiv 2510.13999 · CerebrasResearch/reap
• Chat template: improved variant — a strict superset of the official one (23 message shapes byte-identical, 4 client-crash cases fixed, notably tool_call.arguments sent as a JSON string rather than a mapping).

sha256 for every file is in SHA256SUMS.txt.

Not affiliated with Kwaipilot or Cerebras. Pruning necessarily changes model behaviour; the numbers above are what was measured, and what was not measured is stated as not measured.