Qwen3.8-Flash-Next-REAM-60Pct-GGUF

This is a custom high-quality GGUF build of Akicou/Qwen3.8-Flash-Next-REAM-60Pct.

The goal is simple:

Keep as much model quality as possible while making Qwen3.8-Flash-Next practical on a system with around 64 GiB of RAM.

The final GGUF is approximately:

• 56 GB
• 52 GiB

This build combines:

1. REAM-60Pct
2. High-quality mixed GGUF quantization
3. PLE / n-gram embedding pruning

Instead of using very low-bit quantization on the main model, this build saves a large amount of space by pruning the huge PLE n-gram table. The saved space is then used to keep the main model weights at much higher precision.

Model details

• Architecture: qwen4exp
• Model type: Mixture-of-Experts / MoE
• REAM-compressed model before PLE pruning: about 129B stored parameters
• Active parameters per token: about 6B
• Original PLE / n-gram table: about 51.2B parameters
• Original PLE heads: 16
• PLE heads kept: 4 / 16
• Bigram heads kept: 2 / 8
• Trigram heads kept: 2 / 8
• Approximate PLE parameters physically kept: about 12.8B
• Approximate physically stored parameters after PLE pruning: about 90B
• Final GGUF size: about 56 GB / 52 GiB
• Quantization profile: BALANCED_MAX_PROTECTED

Only around 6B parameters are active for each token because this is a sparse MoE model.

The much larger stored parameter count does not mean all of those parameters are calculated for every token.

Quantization

This build uses the BALANCED_MAX_PROTECTED high-quality mixed quantization profile.

This build does not intentionally use:

• Q2_K
• Q3_K
• Q2_0
• IQ quantization formats

The idea is to spend more of the available memory on higher-quality main model weights instead of compressing everything as aggressively as possible.

PLE / n-gram pruning

Qwen3.8-Flash-Next contains a very large learned PLE n-gram embedding table.

The full table uses:

• 8 bigram hash heads
• 8 trigram hash heads

That gives:

16 PLE heads in total

This build keeps:

• 2 / 8 bigram heads
• 2 / 8 trigram heads

So the final model has:

4 / 16 active PLE heads

The original PLE table contains around 51.2 billion parameters.

Keeping one quarter of the heads leaves roughly:

12.8 billion PLE parameters

physically stored in the compact table.

The other 12 logical heads point to a shared zero row.

This greatly reduces the size of the PLE table while keeping part of the learned n-gram information.

Why prune PLE instead of using Q2/Q3 everywhere?

The PLE table is extremely large, but it is mainly a lookup table.

It is not the same as having another 51B dense transformer parameters being calculated for every token.

By reducing PLE from 16 heads to 4 heads, a large amount of memory can be saved.

That memory can then be used for higher-quality core model weights.

In simple terms:

Less memory is spent on the huge n-gram lookup table, and more memory is spent on the main model.

REAM

This model is based on:

Akicou/Qwen3.8-Flash-Next-REAM-60Pct

REAM reduces the number of stored MoE experts compared with the original Qwen3.8-Flash-Next model.

This lowers the total model size while keeping the sparse Mixture-of-Experts structure.

The model still activates only around 6B parameters per token.

Memory

The final GGUF is approximately:

56 GB / 52 GiB

The model was intentionally kept below the full 64 GiB RAM capacity because llama.cpp needs additional memory for things such as:

• compute buffers
• context
• KV / recurrent state
• memory mapping
• the operating system
• other running programs

This makes a model around 52 GiB much more practical on a 64 GiB RAM machine than a model that uses almost the entire 64 GiB just for its weights.

Actual memory usage depends on context size, llama.cpp settings, GPU offloading, backend, and operating system.

llama.cpp compatibility

This GGUF is designed for a recent upstream version of llama.cpp with Qwen3.8 / qwen4exp support.

No custom llama.cpp fork is required.

The PLE-pruned GGUF still uses the normal Qwen3.8 PLE format expected by llama.cpp.

The 12 removed PLE heads point to a shared zero row.

PLE pruning happens after quantization.

The other quantized model tensors are copied directly into the final GGUF and are not requantized during PLE pruning.

MTP

Qwen3.8-Flash-Next also has a separate MTP draft head in the original model family.

The llama.cpp Qwen3.8 GGUF conversion used for this build does not include the separate MTP draft model.

Because of this:

MTP is not included in this GGUF.

This does not affect normal autoregressive generation.

MTP is mainly useful as a draft model for speculative decoding.

Calibration

This build uses:

• No imatrix
• No calibration dataset
• No retraining
• No LoRA healing

The main model is directly quantized from the BF16 GGUF.

PLE pruning is then performed as a separate GGUF surgery step.

Quality notes

This is an experimental custom quantization.

The main transformer and MoE weights are kept at significantly higher precision than they would be in an aggressive Q2/Q3 build.

However, PLE has been reduced from 16 heads to 4 heads.

PLE can be useful for things such as:

• next-token prediction
• wording and surface-level text prediction
• memorized patterns
• exact phrase prediction

Because of this, PLE pruning can still affect these areas.

The main reasoning model itself is not structurally pruned.

• No transformer layers are removed.
• No additional MoE experts are removed beyond the REAM compression already present in the base model.
• The core model uses relatively high-quality quantization.

The goal of this build is:

Trade some PLE redundancy for substantially higher-quality core model weights while staying practical on a 64 GiB machine.