Sakura — Qwen3.8-Flash-Next, ISTA 3-bit cut with Darwin-R3 tensors, 352 of 512 experts (experimental)

Experimental: no demonstrated advantage over the unmodified cut

In our own coding tests this build scores the same as the unmodified ISTA K352 IQ3_XXS cut within the noise of single runs, and its held-out German/English perplexity is about 3 % higher than the unmodified cut's (see the tables below). It is published for people who want to try the Darwin-R3 weights on the 352-expert layout, not because it was found to be better.

GPU memory: 30.0 GiB dedicated + 11.2 GiB shared after loading (32k context, q8_0 KV, MTP head), about 41 GiB of the Vulkan pool — not for a 32 GB card

It runs on a 64 GB Strix Halo (unified memory). For a 32 GB card use the 2.5-bit sibling Sakura-Qwen3.8-Flash-Next-Swift-2.5bit-352E-32GB-GGUF.

An experimental GGUF of Qwen3.8-Flash-Next: ISTA-DASLab's 3-bit GSQ-RCO quantization cut to 352 of 512 routed experts per layer (chosen on German + English routing, no re-quantization), with 496 non-expert tensors (attention, shared experts, hyper-connection weights) replaced by those of Darwin-180B-RSI-R3 in Q8_0. Size 58.66 GiB in two shards. Independent community build, not an official Qwen, ISTA-DASLab or FINAL-Bench release. Overview of all our Flash-Next builds: Sakura-Qwen3.8-Flash-Next-Overview.

Part of the Sakura Micro line.

Provenance

• Base: Qwen/Qwen3.8-Flash-Next (arch qwen4exp, 48 layers, 512 experts, 10 active).
• Quantization and experts: ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF, IQ3_XXS tier, expert-sliced (see NOTICE.md; kept experts in kept_experts_K352.json).
• Grafted tensors: Darwin-180B-RSI-R3 from FINAL-Bench/POCKET-Darwin-180B-GGUF. According to FINAL-Bench, R3 is a second round of self-training that changes only attention paths and shared experts (LoRA, merged) and leaves the 512 routed experts and the router unchanged; their card reports a small gain on a held-out SuperGPQA set (+1.03 over round 1) and no significant gain on GPQA Diamond (+0.25). We did not reproduce those benchmarks.
• Tools: prune_raw_gguf.py (expert slicing) and graft_tensors.py (tensor graft) are in the Overview repository; the list of grafted tensors is grafted_tensors.json.

Files

SHA256SUMS lists the checksums. Load shard 1; llama.cpp finds shard 2 itself. Use -lm dio --lazy-mode on (or your build's equivalent) so the large token table is read lazily instead of held in memory.

Measured results (own tests; unmodified ISTA K352 IQ3_XXS cut vs this build)

Same runtime for both: Unsloth fork qwen4exp/mtp (commit 6fcaa16, Vulkan, built by us), shared MTP head mtp-Qwen3.8-Flash-Next-shared-Q4_K_M.gguf with n-max 2, -b 2048 -ub 2048, 32k context, KV q8_0, thinking budget 10000, max 12288 tokens, temperature 0.6, top-p 0.95, top-k 20, seeds 1000 + task number, one run each, own frozen task set (LeetCode/AtCoder-style).

Task 15 (the one the unmodified cut fails on tasks 9-20) was repeated with two more seeds on both builds: 0 of 3 on each. The one-task differences elsewhere are inside the noise of single runs.

The language gate and the perplexity were measured with the official llama.cpp build b11259 (Vulkan, no speculation); coding tasks and speed with the fork above. Single runs on one machine; they are indicators, not a benchmark suite. For comparison, the Swift 1.5 fine-tune cut (same architecture and size) reaches about 34.5 tok/s with the same fork and head, also 11/12 on tasks 9-20 and 9/12 on tasks 21-32.

Known limits

• No demonstrated advantage (see the box above). The higher precision of the grafted tensors (Q8_0) and Darwin's training cannot be separated in our tests.
• The Darwin benchmarks (GPQA, AIME, MMLU-Pro) belong to FINAL-Bench's own models and were not measured by us for this build.
• Cutting experts without retraining costs some quality; the size of that cost was checked on German/English text and a small coding set only.
• Speculation with the shared MTP head needs a llama.cpp fork; official llama.cpp runs the file without speculation. Vision was not tested (no mmproj is included).
• Licenses: the weights inherit the Qwen Community License 1.0, included as LICENSE. Read it before commercial use.

Run example

llama-server -m Sakura-Qwen3.8-Flash-Next-ISTA-Darwin-R3-K352-IQ3_XXS-00001-of-00002.gguf --jinja -fa on -ngl 99 -c 32768 -ctk q8_0 -ctv q8_0 -b 512 -ub 512 -lm dio --lazy-mode on --device Vulkan0

(Official llama.cpp b11259, Vulkan, without speculation. For the MTP head use the fork named above.)

English above. 本节为上文的中文翻译(Sakura = 樱花 yīnghuā);完整的独立中文版见 README_zh.md。

Sakura — Qwen3.8-Flash-Next,嫁接了 Darwin-R3 张量的 ISTA 3 比特裁剪版,512 个专家中保留 352 个(实验性)

实验性:相对于未修改的裁剪版本,没有证明有优势

在我们自己的编程测试中,这个构建的得分与未修改的 ISTA K352 IQ3_XXS 裁剪版本 相同(在单次运行的噪声范围内),而且它在留出的德语/英语上的困惑度比未修改的裁剪版本 高约 3 %(见下表)。发布它是为了让想在 352 专家布局上试用 Darwin-R3 权重的人使用,而不是因为发现它更好。

GPU 内存:加载后 30.0 GiB 专用内存 + 11.2 GiB 共享内存(32k 上下文,q8_0 KV,MTP 头),约占 Vulkan 池的 41 GiB — 不适用于 32 GB 显卡

它可以在 64 GB 的 Strix Halo(统一内存)上运行。对于 32 GB 的显卡,请使用 2.5 比特的同系列版本 Sakura-Qwen3.8-Flash-Next-Swift-2.5bit-352E-32GB-GGUF。

这是 Qwen3.8-Flash-Next 的一个实验性 GGUF:ISTA-DASLab 的 3 比特 GSQ-RCO 量化,裁剪为每层 512 个路由专家中保留 352 个(依据德语 + 英语的路由情况选出,没有重新量化), 并把 496 个非专家张量(注意力、共享专家、超连接权重)替换为 Darwin-180B-RSI-R3 的对应张量(Q8_0)。大小 58.66 GiB,分为两个分片。 这是独立的社区构建版本,不是 Qwen、ISTA-DASLab 或 FINAL-Bench 的官方发布。我们所有 Flash-Next 构建的概览:Sakura-Qwen3.8-Flash-Next-Overview。

属于 Sakura Micro 系列。

来源

• 基础模型:Qwen/Qwen3.8-Flash-Next(架构 qwen4exp,48 层,512 个专家,激活 10 个)。
• 量化和专家:ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF,IQ3_XXS 档位,已进行专家切片(见 NOTICE.md;保留的专家见 kept_experts_K352.json)。
• 嫁接的张量:来自 FINAL-Bench/POCKET-Darwin-180B-GGUF 的 Darwin-180B-RSI-R3。据 FINAL-Bench 称,R3 是第二轮自训练,只改变注意力路径和共享专家(LoRA,已合并),512 个路由专家和路由器保持不变;他们的卡片报告了在留出的 SuperGPQA 集上有小幅提升(比第一轮高 +1.03),而在 GPQA Diamond 上没有显著提升(+0.25)。我们没有复现这些基准测试。
• 工具:prune_raw_gguf.py(专家切片)和 graft_tensors.py(张量嫁接)在 Overview 仓库中;被嫁接张量的列表是 grafted_tensors.json。

文件

SHA256SUMS 列出了校验和。加载分片 1;llama.cpp 会自己找到分片 2。请使用 -lm dio --lazy-mode on(或你所用构建中的等效选项),使巨大的 token 表被懒加载读取,而不是常驻内存。

测量结果(自己的测试;未修改的 ISTA K352 IQ3_XXS 裁剪版本 对比 本构建)

两者使用相同的运行时:Unsloth 分支 qwen4exp/mtp(提交 6fcaa16,Vulkan,由我们自己构建),共享 MTP 头 mtp-Qwen3.8-Flash-Next-shared-Q4_K_M.gguf,n-max 2,-b 2048 -ub 2048,32k 上下文,KV q8_0,思考预算 10000,最多 12288 个 token,temperature 0.6,top-p 0.95,top-k 20,随机种子 1000 + 任务编号,各运行一次,使用自己的冻结任务集(LeetCode/AtCoder 风格)。

任务 15(未修改的裁剪版本在任务 9-20 中失败的那一个)在两个构建上又用另外两个随机种子重复了一次:各自都是 3 次中 0 次通过。其他地方相差一个任务的差异处于单次运行的噪声范围内。

语言关卡和困惑度是用官方 llama.cpp 构建 b11259(Vulkan,无推测解码)测得的;编程任务和速度使用上述分支。单次运行,在一台机器上;它们是指示性的,不是基准测试套件。 作为对比,Swift 1.5 微调裁剪版本(相同架构和大小)在相同的分支和头下达到约 34.5 tok/s,在任务 9-20 上同样是 11/12,在任务 21-32 上是 9/12。

已知局限

• 没有证明有优势(见上方的方框)。在我们的测试中,无法把被嫁接张量更高的精度(Q8_0)与 Darwin 的训练区分开。
• Darwin 的基准(GPQA、AIME、MMLU-Pro)属于 FINAL-Bench 自己的模型,我们 没有 为这个构建测量过。
• 不经再训练就裁剪专家会损失一些质量;这一损失的大小只在德语/英语文本和一个小型编程集上检查过。
• 与共享 MTP 头一起使用的推测解码需要 llama.cpp 分支;官方 llama.cpp 在没有推测解码的情况下运行该文件。视觉能力未经测试(未包含 mmproj)。
• 许可证:权重继承 Qwen Community License 1.0,以 LICENSE 随附。商业使用之前请先阅读。

运行示例

llama-server -m Sakura-Qwen3.8-Flash-Next-ISTA-Darwin-R3-K352-IQ3_XXS-00001-of-00002.gguf --jinja -fa on -ngl 99 -c 32768 -ctk q8_0 -ctv q8_0 -b 512 -ub 512 -lm dio --lazy-mode on --device Vulkan0

(官方 llama.cpp b11259,Vulkan,无推测解码。若要使用 MTP 头,请使用上面提到的分支。)