Qwen3.8-Flash-Next REAP-288 (GGUF)
This is the REAP-288 build in GGUF, for llama.cpp and the tools built on it (Ollama, LM Studio, and similar). It is the 180B-class MoE with its experts pruned 512 -> 288 per layer, converted directly from the bf16 source in a single quantization step, so each file loses accuracy exactly once rather than stacking a re-quantization on top of an existing 4-bit build.
The pruned model scores 91.5% on HumanEval (149 of 164), against 93.9% for the unpruned full model. That number is measured on the MLX 4-bit lineage; the GGUF quants share the same kept-expert selection and derive from the same bf16 source.
If you run MLX on Apple Silicon instead of llama.cpp, use the 4-bit MLX build. The pmlx engine and its speedups apply to the MLX builds, not to GGUF; nothing on this card depends on it.
Files
Converted from the bf16 source. Pick one quant; larger files are higher
fidelity and slower. Take Q4_K_M for the smallest, fastest build, Q8_0 for
the highest quality, and Q5_K_M if you want a middle point.
| Quant | Size | Notes |
|---|---|---|
Q4_K_M | 78 GB | good default: smallest and fastest, strong quality |
Q5_K_M | 87 GB | higher fidelity, more memory |
Q8_0 | 116 GB | near-lossless, largest and highest quality |
Run it
llama.cpp:
# build llama.cpp, then:
llama-server -m Qwen3.8-Flash-Next-REAP-288-Q4_K_M.gguf --port 8080
# or a one-shot completion:
llama-cli -m Qwen3.8-Flash-Next-REAP-288-Q4_K_M.gguf -p "Refactor this function to add input validation."
Ollama (pulls the quant straight from this repo):
ollama run hf.co/sh0wie/Qwen3.8-Flash-Next-REAP-288-GGUF:Q4_K_M
Provenance
Qwen/Qwen3.8-Flash-Next: upstream weights.- sh0wie/Qwen3.8-Flash-Next-REAP-288-bf16: the full-precision REAP-288 source these files are quantized from. REAP expert pruning 512 -> 288 per layer, calibrated on-device over ~686K tokens of agentic-coding traffic.
Limitations
- Calibration reflects one team's agentic-coding distribution. Retention numbers should not be read as general-domain; domains far from code may degrade more.
- Single-run evaluations on the lineage, no confidence intervals. Vision input is untested after pruning.
License
Qwen Community License 1.0, inherited from the base model Qwen/Qwen3.8-Flash-Next.