Swift-Qwen3.8-27B NVFP4 GGUF
A llama.cpp GGUF of ukisai/Swift-Qwen3.8-27B-NVFP4, UkisAI's calibrated NVFP4 checkpoint of Swift-Qwen3.8-27B (a reasoning-efficient fine-tune of Qwen3.8-27B). UkisAI publish that NVFP4 checkpoint for vLLM only; this file brings it to llama.cpp without requantizing any of its NVFP4 weights.
| File | Size | BPW |
|---|---|---|
Swift-Qwen3.8-27B-NVFP4-Q8mix.gguf | 19.72 GB | 5.77 |
Tensor layout
| Tensors | Type | Source |
|---|---|---|
| All MLP (64 layers × gate/up/down) + full attention q/k/v/o (16 layers) — 256 tensors | NVFP4 | UkisAI's llm-compressor NVFP4, byte-identical to the checkpoint (with per-tensor scales) |
Gated DeltaNet linear attention (attn_qkv, attn_gate, ssm_out, 48 layers) | Q8_0 | BF16 in the checkpoint |
output (lm_head), token_embd | Q8_0 | BF16 in the checkpoint |
MTP head (blk.64, incl. nextn.eh_proj) | Q8_0 | BF16 in the checkpoint |
ssm_alpha, ssm_beta, ssm_conv1d | F32 | BF16 in the checkpoint (lossless) |
The MTP head is included, so --spec-type draft-mtp works.
How it was made
With the official ghcr.io/ggml-org/llama.cpp:full image (build b10975):
convert_hf_to_gguf.py --outtype bf16on the checkpoint. It has a singlenvfp4-pack-quantizedconfig group, which the converter repacks natively as GGML NVFP4.llama-quantizewith an exact per-tensor--tensor-type-file: NVFP4 tensors map tonvfp4, which llama-quantize copies unchanged when source and target types match; the BF16 leftovers go to Q8_0, the recurrent gates to F32.- Verified: all 256 NVFP4 tensors and their 512 scale tensors are SHA-256 identical to the
conversion of the checkpoint, and all 20
qwen35.*architecture metadata keys match UkisAI's own GGUF of Swift.
Usage
llama-server -m Swift-Qwen3.8-27B-NVFP4-Q8mix.gguf \
--mmproj mmproj-Swift-Qwen3.8-27B-F16.gguf \
--spec-type draft-mtp --spec-draft-n-max 3 \
-c 262144 -fa on --jinja
- NVFP4 runs natively on NVIDIA Blackwell. Other GPUs need a llama.cpp build with an NVFP4 fallback path.
- Vision: use the projector from
ukisai/Swift-Qwen3.8-27B-GGUF
(
mmproj-Swift-Qwen3.8-27B-F16.gguf). - Chat template:
reasoning_effortacceptsxhigh(default),medium,low. Unlike some Qwen templates, it does not accepthigh. - Sampling (stored in the GGUF header): temperature 1.0, top_p 0.95, top_k 20, min_p 0.
Quality
Not yet measured for this file. UkisAI report the source NVFP4 checkpoint within sampling noise of BF16 on their vLLM harness (MTP acceptance 61% vs 62%). For reference, UkisAI's own GGUF tiers measure KLD vs BF16 of 0.0120 (Q4_K_M), 0.0056 (Q5_K_M), 0.0020 (Q6_K), 0.0009 (Q8_0) on wikitext @512.
License
Swift weights are under the Swift Open License v1.0: free for personal, research, educational, evaluation and commercial use by individuals and organizations with annual recurring revenue (including affiliates) up to US$1,000,000. Above that, commercial use requires a Swift Enterprise License from UkisAI. See the source model card.
All credit for the model and the NVFP4 calibration goes to UkisAI; the base model is Qwen/Qwen3.8-27B by the Qwen team.