Qwen3.8 Flash Next — Ombro V9 GGUF

Ombro V9 is an experimental, measurement-guided mixed-precision GGUF quantization of Qwen/Qwen3.8-Flash-Next. Where the earlier Ombro Q1 release traded precision for size and speed, V9 is quality-first: every routed-expert precision choice had to hold a strict local error floor before any bytes were saved.

Orientation: V9 is built around Unsloth's UD quants of this model. It takes UD-Q4_K_XL (111.3 GB) as its quality floor and beats it: no routed-expert layer is less accurate than UD-Q4_K_XL on any of the local error measures below, and the extra bytes go where the measurements show they matter. At 114.5 GB it stays far below UD-Q5_K_XL (158.3 GB).

Ombro is a community quantization recipe, not an official GGUF quant type. This repository is not affiliated with Qwen, Unsloth, Bartowski, or llama.cpp.

Download

Download all nine files. Point llama.cpp at the first shard; the remaining shards are discovered automatically.

Checksums are in SHA256SUMS.

Quantization approach

The official BF16 Safetensors checkpoint was converted to a source-faithful BF16 GGUF and verified tensor-by-tensor before quantization.

V9 then chooses the storage type of the routed-expert gate, up, and down projections independently in each of the 48 layers. For every layer, up to 108 gate/up/down type combinations were measured against BF16 expert outputs on recorded real activations and router decisions, across two separate calibration partitions, using 13 error measures (expert and routed NMSE, tail and worst-token NMSE, output direction and output norm, and worst-expert NMSE).

A combination was only eligible if each of those 13 measures, individually and on both partitions, was no worse than the corresponding layer's Unsloth UD-Q4_K_XL type combination measured the same way. No average was allowed to hide a single regression. (One layer, for which the UD combination was not measured, used an earlier Ombro reference as its floor instead.) Among eligible combinations, an exact byte-budget optimizer distributed the remaining precision.

Routed experts per layer (gate / up / down), compared with Unsloth's UD-Q4_K_XL and UD-IQ3_XXS. Only layers that differ from the common pattern are listed:

Experimental follow-ups (not released)

Built from the same BF16 source and the same importance matrix, to test single changes against V9. Layers not listed are as in V9 (Q4_K / Q4_K / Q5_1).

The down projections have 640-wide rows, which is not divisible by the 256-element blocks of K-quants, so only 32-element block types are possible there. V9 uses Q5_1 as the floor everywhere, with Q8_0 on six layers. There is no Q5_0 in this model.

The per-layer token embedding table (28.8 GB, a quarter of the model) is IQ4_NL; its 160-wide rows admit no smaller block format. Small control tensors, for example ssm_alpha, ssm_beta, the hyper-connection inject weights and the layer-1 PLE convolution, are kept at F32. Token embeddings, output weights, and unlisted always-active tensors use Q8_0 where their source type permits it. The complete per-tensor override map is included as quantization-recipe.txt.

The importance matrix combines the public Qwen3.8-Flash-Next matrices from Unsloth and Bartowski by raw accumulator summation. The merged matrix contains 926 entries and represents 2,234 calibration chunks of 512 tokens. Its SHA-256 is 7e8c06b000b3a4146997a5db81d651709aee12694b256be0832507460af287ff.

The quantization used llama.cpp commit bbdd9f246e9667f9aeb7ad11cca269466f981184. The final split contains 1,224 unique tensors.

What the local measurements do and do not say

The selection rule guarantees a local property: per layer, the chosen routed expert types reproduce BF16 expert outputs at least as well as the UD-Q4_K_XL type combination, on every one of the 13 measures, on the recorded activations. It does not by itself guarantee end-to-end behaviour, because errors propagate between layers and the comparison uses the UD type combination measured locally rather than the published UD file.

Performance

Throughput figures will be added after a controlled sweep. A first local run was not measured under comparable conditions and is intentionally not reported here.

Running with llama.cpp

Use a recent CUDA-enabled llama.cpp build with Qwen3.8-Flash-Next support. A starting point for a two-GPU system is:

llama-server \
  --model Qwen3.8-Flash-Next-Ombro-v9-00001-of-00009.gguf \
  --jinja \
  --ctx-size 131072 \
  --parallel 1 \
  --flash-attn on \
  --cache-type-k bf16 \
  --cache-type-v bf16 \
  --load-mode mmap \
  --fit on \
  --batch-size 2176 \
  --ubatch-size 768 \
  --cache-reuse 256 \
  --no-context-shift \
  --temp 1.0 \
  --top-p 0.95 \
  --top-k 20 \
  --min-p 0.0 \
  --repeat-penalty 1.0

Reduce batch sizes if your available VRAM is lower. For this model's chat template, valid reasoning efforts are low, medium, and xhigh; unsupported labels such as high are rejected by llama-server.

--cache-reuse helps agentic and tool-calling workloads, where tool output inserted between turns makes the prompt prefix diverge. In image conversations, the cache is invalidated from the first image onward regardless.

Vision and MTP

These shards contain the main language model only. The base model's vision encoder is available separately as the Unsloth mmproj-F16.gguf, which has been used with these shards locally (--mmproj).

The base model has an MTP architecture, but these shards contain no nextn/MTP tensors. Speculative decoding needs a compatible external MTP draft model and a llama.cpp build that supports MTP for this architecture.

Validation status

• Official BF16 source conversion verified across all 1,224 tensors.
• Final size matches the pre-computed byte plan to within 304 bytes.
• Every entry of the published override map was confirmed in the quantizer log; no quantizer fallback occurred.
• Loads and generates successfully with llama.cpp, including image input with the Unsloth mmproj.
• No formal perplexity, KL-divergence, long-context accuracy, coding, agentic, or broad source-equivalence evaluation has been completed yet.

License and attribution

The original model is released under the Qwen Community License 1.0. The included LICENSE is copied from the official model repository. Review the original model card, intended use, limitations, and license terms before using or redistributing this model.