Qwen 3.6 35B A3B REAP IQ1_M GGUFs

This repo contains three experimental IQ1_M GGUF quants derived from REAP-pruned Qwen 3.6 35B A3B checkpoints. I just wanted to get a local LLM that could answer the car wash test correctly on my iPad Pro with 8GB RAM as all the other smaller LLMs it could run just couldn't, and also faster than 3tk/s for the full 35B IQ1M model due to partial offload to swap.

These are not plain base-model quants. They come from REAP-pruned MoE checkpoints, then were converted to trunk-only GGUF and quantized with a Bartowski-style Qwen 3.6 IQ1_M tensor policy.

What These Are

Starting point:

• Qwen/Qwen3.6-35B-A3B

Pruning method:

• REAP expert pruning on the HF BF16 model

Quantization method:

• GGUF export
• trunk-only / no-mtp conversion path
• fresh per-model imatrix
• Bartowski-style Qwen 3.6 IQ1_M tensor map

Recommended Variant

• Use qwen3.6-35b-reap25_iq1m.gguf if you want the strongest general-purpose REAP variant.
• Use qwen3.6-35b-reap40_iq1m.gguf if you care more about instruction-following and strict prompt obedience.
• Use qwen3.6-35b-reap50_iq1m.gguf only if size is the main priority.

Benchmarks

Interpretation

• reap25 is the strongest broad-quality REAP quant.
• reap40 is the most interesting instruction-following variant. It gives up a lot on HellaSwag, but beats reap25 and reap50 on both strict IFEval metrics.
• reap50 is the most compressed and the most uneven. It keeps ARC surprisingly well, but broad completion quality drops much harder.

One likely explanation is that heavier REAP pruning narrows the model more than it destroys literal instruction compliance. That can hurt broad reasoning / continuation tasks while still preserving, or sometimes slightly improving, rigid format-following behavior.