KAT-Coder-V2.5-Dev · REAP-50 (pruned bf16)
The 50%-REAP-pruned Kwaipilot/KAT-Coder-V2.5-Dev
in bf16 — the source checkpoint behind the NVFP4 and GGUF releases. Provided so
others can produce their own quants (AWQ, EXL2, MLX, custom GGUF, …).
- 256 → 128 experts via REAP (Router-weighted Expert Activation Pruning), with a router-renormalization fix over the survivors.
qwen3_5_moehybrid (Gated-DeltaNet + attention + MoE). No MTP head (mtp_num_hidden_layers: 0); reload-verified with a real forward pass.- Vision tower not stripped in this checkpoint — use
Qwen3_5MoeForCausalLMfor text-only.
Base-model quality (measured on the NVFP4A16 quant of this checkpoint, greedy, instruct): HumanEval+ ~90%, MBPP+ ~90%. These are single greedy draws; the accuracy harness has a measured run-to-run spread of 1.85–4.27 pp per task (2026-09-05), so read small differences as unresolved.
Architecture
graph TD
Base["Kwaipilot/KAT-Coder-V2.5-Dev<br/>Qwen3.5 MoE - 256 experts - ~69 GB bf16"]
subgraph Build ["Build - RTX 5070 Ti, SM120"]
REAP["REAP expert prune 50% 256 -> 128 experts + router-renorm fix"]
Strip["strip vision tower + 333 untrained tensors"]
Quant["NVFP4A16 quantize (weight-only, data-free, 82 s)"]
end
subgraph HF ["Published formats"]
A16["REAP-50-NVFP4A16 - 12.45 GiB (default, vLLM)"]
W4A4["REAP-50-NVFP4-W4A4 (native FP4)"]
GPTQ["REAP-50-NVFP4A16-GPTQ (null result, kept for transparency)"]
GGUF["REAP-50-GGUF (Q4_K_M / Q5_K_M / Q6_K / Q8_0)"]
BF16["REAP-50-bf16 (pruned source)"]
end
Bench["A16 single draws - HumanEval+ 89.0% - MBPP+ 90.5% - SWE-bench Verified 52.0%"]
Base --> REAP --> Strip --> Quant --> A16
Strip --> BF16
BF16 -. re-quant .-> W4A4
BF16 -. re-quant .-> GPTQ
BF16 -. convert .-> GGUF
A16 --> Bench
Releases built from this
- NVFP4A16 (vLLM / Blackwell):
Ttimms/KAT-Coder-V2.5-Dev-REAP-50-NVFP4A16 - NVFP4 W4A4:
Ttimms/KAT-Coder-V2.5-Dev-REAP-50-NVFP4-W4A4 - GGUF:
Ttimms/KAT-Coder-V2.5-Dev-REAP-50-GGUF
Pipeline
Full prune → quant → serve → evaluate pipeline: https://github.com/t-timms/kat-coder-nvfp4
License
Apache-2.0 (inherits from Kwaipilot/KAT-Coder-V2.5-Dev). Pruning via REAP
(github.com/CerebrasResearch/reap, with a router-renormalization fix).