Qwen3.8-27B EfficientThink · SimPO · KQV-Q4 + IQ3 Floor · MTP
English ｜ 中文（点击跳转）
Qwen3.8 27B is a great model. Capable, but it eats VRAM: BF16 wants ~54 GB, Q6_K still wants 20.8 GiB, a typical Q3 build sits at 12-13 GB, and on a 16 GB card the context budget is simply too tight. Most GGUFs out there take the uniform-quantization road, straight down to Q3 territory — the size isn't really reduced by much, and the quality is just mediocre. I've gone through a fair number of other people's quantized builds, and none of them satisfied me.
So I hand-picked the base model, and quantized with focus and selectivity, using a mixed-precision strategy I worked out on my own. This is the KQV-Q4 + IQ3 Floor build of the SimPO Q6_K weights: 20.79 GiB down to 10.12 GiB (-51%), without letting anything that matters drop — every K/Q/V anchor block stays at Q4_K or above, every norm stays F32, and the FFN layers ride the IQ3_XXS floor, calibrated with an importance matrix. Built-in MTP. The source is Q6_K, so the quality is guaranteed. On a 16 GB card with a Q4 KV cache, 130k+ context still fits.
What sets it apart: quantized straight from Q6_K — the SimPO post-training distribution is the direct source, with no BF16 round trip; the full 15-tensor MTP head; 130k+ context on 16 GB. All of it in one 10 GB file.
Built on Qwen3.8; thanks to the Qwen team, and to nerkyor for the excellent base. This file is that family's SimPO / Q6_K branch: grafted with MTP, then mixed-precision quantized.
Files
| File | Size | Role |
|---|---|---|
Qwen3.8-27B-EfficientThink-SimPO-KQVQ4-IQ3Floor-10G-MTP.gguf | 10,868,156,096 B (10.12 GiB) | This model |
SHA256 of this model: 80e3f758f8bca53b07754d80ceb5310df74d18e9d2b1d7d3925a9f5dda9ae9b9
Mixed precision: KQV-Q4 + IQ3 Floor
866 tensors, 3.18 BPW. The allocation was extracted block-by-block from the accepted production 10 GB build:
| Role | Type | Tensors | Why |
|---|---|---|---|
| K/Q/V of the 16 full-attention anchor blocks (blk.3, 7, …, 63) | Q4_K | 48 | The anchors keep the quality — that's the KQV promise |
MTP head blk.64.attn_v | Q4_K | 1 | The draft value path stays high-precision |
| All FFN + anchor attn_output + linear attn_gate/qkv | IQ3_XXS | 327 | The FFN rides the 2.06 BPW floor — this is where the size savings come from |
| SSM α/β (48×2) + ssm_out blk.30+ + MTP head weights + token_embd + output | IQ3_S | 130 | The sequence-state paths, one step above the floor |
| All norms + ssm_a / ssm_conv1d / ssm_dt.bias | F32 | 360 | No degradation where it costs nothing |
Type histogram: Q4_K 49 · IQ3_S 130 · IQ3_XXS 327 · F32 360 — identical to the production pattern, verified per tensor. The IQ3-level tensors were quantized with the unsloth importance matrix (1,251 chunks × 8,192 tokens, 496 weight entries covering blk.0-63); the ten MTP-head and embedding tensors without matrix entries use standard quantization, exactly like the production model.
Why this beats the plain 10 GB tier
- Half the size, full anchors — 20.79 → 10.12 GiB (-51%) while every K/Q/V anchor stays at Q4_K and every norm at F32.
- The MTP head is inside — some 10 GB builds ship with MTP, some don't; this one carries the complete 15-tensor head and runs
--spec-type draft-mtpat roughly 2-3× on chat-length outputs. - 130k+ context on 16 GB — a same-size f16-KV build gives you a fraction of that.
MTP
block_count = 65,nextn_predict_layers = 1— the MTP head is a full decoder block atblk.64(11 attention/FFN tensors + 4nextntensors).- Head precision:
attn_v= Q4_K, the remaining weights = IQ3_S (one step above the floor), all norms F32. - Draft quality inherits the Q6_K distribution — and with it, the acceptance rate.
Usage
Tested on llama.cpp b10728:
llama-server.exe `
-m Qwen3.8-27B-EfficientThink-SimPO-KQVQ4-IQ3Floor-10G-MTP.gguf `
-c 131072 -ngl 99 -t 8 --kv-unified `
--jinja --reasoning on --reasoning-format deepseek `
--spec-type draft-mtp -np 1 --spec-draft-n-max 2 --spec-draft-p-min 0 `
--cache-type-k q4_0 --cache-type-v q4_0 -fa on `
--top-k 20 --top-p 0.95 --repeat-penalty 1.1 --min-p 0
--spec-type draft-mtp— turns on the built-in MTP head (expect ~2-3× on chat-length outputs).--cache-type-k q4_0 --cache-type-v q4_0— Q4 KV cache; this is what lets 130k fit in 16 GB.- For vision, pair it with the Qwen3.8-27B vision projector from the base family via
--mmproj. - LM Studio: import the GGUF directly and enable speculative decoding (MTP) in the server settings.
Verification
| Check | Result |
|---|---|
| dry-run size | 10,354.20 MiB (3.18 BPW) |
| Real output | 10,868,156,096 B — 0.00% deviation vs dry-run |
| Tensor count | 866 = 851 main + 15 MTP head |
| MTP metadata | block_count=65, nextn_predict_layers=1 |
| Type histogram | Q4_K 49 / IQ3_S 130 / IQ3_XXS 327 / F32 360 |
| Per-tensor pattern | all 64 main blocks + MTP head + base tensors — zero deviation |
Disclaimer
This model is provided AS IS, without warranty of any kind, express or implied; the Apache 2.0 license terms apply. It is a personal research build — intended for individual research and evaluation, not for public or production deployment. As an uncensored mixed-precision build, outputs are unpredictable: you are responsible for your own prompts and outputs. Test thoroughly in your own environment before relying on anything it generates; all risk is yours, and the author assumes no liability for any loss.
Base model: nerkyor/Qwen3.8-27B-EfficientThink-Uncensored-K3-Opus5-Grok4.6-GPT5.6Sol-SFT-SimPO-DFlash2.
中文说明
Qwen3.8 27B 是一个伟大的模型。能力够，但是显存也吃得凶：BF16 要 ~54 GB，Q6_K 也要 20.8 GiB，通常的 Q3 要 12G 到 13G，一张 16 GB 的卡，上下文太紧缺了。而且市面上的 GGUF，大多走均匀量化、压到 Q3 那一档——体积其实没有压缩多少，质量还一般。我也看过不少别人量化的版本，始终没有一款令我满意的。
所以精选了底模，又用自己探索出的混合量化策略，有重点、有选择地进行量化。这就是 SimPO Q6_K 权重的 KQV-Q4 + IQ3 Floor 混合精度版：20.79 GiB 压到 10.12 GiB（-51%），该保的地方一处不降——每个 K/Q/V 锚点块不低于 Q4_K，每个 norm 保持 F32，FFN 走 IQ3_XXS 地板并用重要性矩阵校准。内置 MTP。源模型是 Q6_K，质量有保证。16 GB 显卡 + Q4 KV 缓存，13 万+ 上下文照样装得下。
它的不同之处：直接从 Q6_K 量化——SimPO 的后训练分布就是量化源，不绕 BF16 中转；带完整 15 张量 MTP 头；16 GB 开 13 万+ 上下文。都装在一个 10 GB 的文件里。
基于 Qwen3.8；感谢 Qwen 团队，感谢底模作者 nerkyor。 本文件即该家族 SimPO / Q6_K 分支：移植 MTP 后做混合精度量化。
文件
| 文件 | 大小 | 角色 |
|---|---|---|
Qwen3.8-27B-EfficientThink-SimPO-KQVQ4-IQ3Floor-10G-MTP.gguf | 10,868,156,096 B（10.12 GiB） | 本模型 |
本模型 SHA256：80e3f758f8bca53b07754d80ceb5310df74d18e9d2b1d7d3925a9f5dda9ae9b9
混合精度：KQV-Q4 + IQ3 Floor
866 张量，3.18 BPW。分配表从已验收的生产 10 GB 构建逐块提取：
| 角色 | 类型 | 张量数 | 理由 |
|---|---|---|---|
| 16 个全注意力锚点块的 K/Q/V（blk.3, 7, …, 63） | Q4_K | 48 | 锚点保质量——这就是 KQV 的承诺 |
MTP 头 blk.64.attn_v | Q4_K | 1 | 草稿 value 路径保高精度 |
| 全部 FFN + 锚点 attn_output + 线性 attn_gate/qkv | IQ3_XXS | 327 | FFN 走 2.06 BPW 地板——体积节省在此 |
| SSM α/β（48×2）+ ssm_out blk.30+ + MTP 头权重 + token_embd + output | IQ3_S | 130 | 序列状态路径，地板上一档 |
| 全部 norm + ssm_a / ssm_conv1d / ssm_dt.bias | F32 | 360 | 零代价处零降级 |
类型直方图 Q4_K 49 · IQ3_S 130 · IQ3_XXS 327 · F32 360 与生产模式一致，逐张量校验通过。IQ3 级张量用 unsloth 重要性矩阵量化（1,251 chunks × 8,192 tokens，496 条覆盖 blk.0-63）；无矩阵条目的 10 个 MTP/嵌入张量走标准量化，与生产模型完全相同。
相对普通 10 GB 档的优势
- 体积减半、锚点全保 —— 20.79 → 10.12 GiB（-51%），K/Q/V 锚点全部 ≥ Q4_K，norm 全部 F32。
- MTP 头在里面 —— 同档有的带 MTP、有的不带；这个带完整 15 张量头部，
--spec-type draft-mtp在聊天长度输出上约 2-3×。 - 16 GB 开 13 万+ 上下文 —— 同体积 f16-KV 构建只有零头。
MTP
block_count = 65、nextn_predict_layers = 1——MTP 头是blk.64完整块（11 个注意力/FFN 张量 + 4 个nextn张量）。- 头部精度：
attn_v= Q4_K，其余权重 = IQ3_S（地板上一档），norm 全部 F32。 - 草稿质量继承 Q6_K 分布，接受率也随之而来。
用法
llama.cpp b10728 实测：
llama-server.exe `
-m Qwen3.8-27B-EfficientThink-SimPO-KQVQ4-IQ3Floor-10G-MTP.gguf `
-c 131072 -ngl 99 -t 8 --kv-unified `
--jinja --reasoning on --reasoning-format deepseek `
--spec-type draft-mtp -np 1 --spec-draft-n-max 2 --spec-draft-p-min 0 `
--cache-type-k q4_0 --cache-type-v q4_0 -fa on `
--top-k 20 --top-p 0.95 --repeat-penalty 1.1 --min-p 0
--spec-type draft-mtp——启用内置 MTP 头（聊天长度输出预期 ~2-3×）。--cache-type-k q4_0 --cache-type-v q4_0——Q4 KV 缓存；13 万装进 16 GB 靠的就是它。- 多模态：用
--mmproj配底模家族的 Qwen3.8-27B 视觉投影器。 - LM Studio：直接导入 GGUF，服务端设置里启用投机解码（MTP）。
验证
| 项目 | 结果 |
|---|---|
| dry-run 体积 | 10,354.20 MiB（3.18 BPW） |
| 实量化体积 | 10,868,156,096 B——偏差 0.00% |
| 张量数 | 866 = 851 主干 + 15 MTP 头 |
| MTP 元数据 | block_count=65、nextn_predict_layers=1 |
| 类型直方图 | Q4_K 49 / IQ3_S 130 / IQ3_XXS 327 / F32 360 |
| 逐张量模式 | 64 主干块 + MTP 头 + 基础张量——零偏差 |
免责声明
本模型按原样（AS IS）提供，不作任何明示或暗示的保证；以 Apache 2.0 许可证条款为准。本构建为个人研究用途——供个人研究与评估，不适合公共或生产部署。作为**无审核（uncensored）**混合精度构建，输出不可预期：使用者对自己的提示与输出负责。请在自己的环境中充分测试后再决定是否依赖其生成内容——风险自负，作者不对任何损失承担责任。
底模：nerkyor/Qwen3.8-27B-EfficientThink-Uncensored-K3-Opus5-Grok4.6-GPT5.6Sol-SFT-SimPO-DFlash2。
底模权重与许可证归 nerkyor。MTP 移植与 KQV-Q4 + IQ3 Floor 混合精度量化为本构建新增。 Base weights and license by nerkyor, with thanks. MTP graft and mixed-precision quantization added by this build.