Swift-Qwen3.8-27B-Genesis — NVFP4 GGUF
NVFP4 quantisation of LuffyTheFox/Swift-Qwen3.8-27B-Genesis-F16-GGUF, built with advanced-gguf-quantizer (a llama.cpp fork focused on NVFP4 quantization).
Base model: ukisai/Swift-Qwen3.8-27B-GGUF with Genesis tensor repair by LuffyTheFox.
Files
v4 — Recommended (inline scales, LM Studio compatible)
| File | Calibration | MTP | Size | Tensors |
|---|---|---|---|---|
Swift-Qwen3.8-27B-Genesis-NVFP4-v4.gguf | none (data-free) | yes (blk.64 preserved) | 15.76 GB | 866 |
Swift-Qwen3.8-27B-Genesis-NVFP4-v4-noMTP.gguf | none (data-free) | no (blk.64 stripped) | 15.53 GB | 851 |
v4 uses --nvfp4-inline-scales-only to emit native inline UE4M3 scales
without separate .scale/.input_scale tensors. Required for LM Studio /
Pelican and other runtimes that do not support the extended NVFP4 scale
tensor contract.
Imatrix variants will follow after the plain v4 is confirmed working in target runtimes.
Source
| Source GGUF | Swift-Qwen3.8-27B-F16-0000{1..3}-of-00003.gguf (55.6 GB total, split) |
| Architecture | qwen35 (dense), 64 layers + 1 MTP/NextN block |
| Layers | 48 Gated DeltaNet + 16 gated-attention |
general.file_type | 39 (LLAMA_FTYPE_MOSTLY_NVFP4) |
| MTP/NextN | qwen35.nextn_predict_layers=1 (native, preserved) |
Single-step F16 → NVFP4 — the F16 source gives the NVFP4 encoder clean data
with no intermediate quantization noise. llama-quantize reads the split
source directly via ...-00001-of-00003.gguf.
Tensor mix (v4)
| type | count | notes |
|---|---|---|
| F32 | 360 | norms, ssm scalars (a/dt/conv1d), nextn norms |
| F16 | 4 | blk.0 attn_gate/attn_qkv, blk.0/blk.13 ffn_down |
| NVFP4 | 502 | bulk weights incl. output.weight, token_embd |
| total | 866 | |
separate .scale/.input_scale | 0 | inline UE4M3 only |
Tensor protection policy
F16 singular-collapse protection:
| tensor | type |
|---|---|
blk.0.attn_gate.weight | F16 |
blk.0.attn_qkv.weight | F16 |
blk.0.ffn_down.weight | F16 |
blk.13.ffn_down.weight | F16 |
F32 architecture-specific protection:
blk.*.attn_norm.weight,blk.*.post_attention_norm.weightblk.*.attn_q_norm.weight,blk.*.attn_k_norm.weightblk.*.ssm_norm.weight,blk.*.nextn.*.norm.weightoutput_norm.weightblk.*.ssm_conv1d.weight,blk.*.ssm_dt.bias,blk.*.ssm_a
Forced NVFP4 (do not push lower):
| tensor | type |
|---|---|
blk.0.ssm_out.weight | NVFP4 |
blk.1.attn_gate.weight | NVFP4 |
blk.1.attn_qkv.weight | NVFP4 |
Usage
llama-cli -m Swift-Qwen3.8-27B-Genesis-NVFP4-v4.gguf \
--mmproj mmproj-Swift-Qwen3.8-27B-F16.gguf \
--jinja -c 131072 -ngl 99
For the MTP variant, use Swift-Qwen3.8-27B-Genesis-NVFP4-v4-noMTP.gguf when
the runtime lacks FastMTP support, or the MTP file for speculative decoding.
- Set K cache and V cache quantization to F16
- Vision support requires the
mmproj-Swift-Qwen3.8-27B-F16.gguffrom the source repository
Hardware
- Blackwell (RTX 50xx): native FP4 path, fastest
- Ampere (RTX 30xx): NVFP4 inference works via fallback kernels
- Quantisation was done CPU-only (Ampere CUDA NVFP4 encoder is unreliable)
Reproducibility
# F16 split source -> NVFP4 v4 (inline scales only, CPU-only)
CUDA_VISIBLE_DEVICES="" llama-quantize \
--allow-requantize --mode fast --nvfp4-inline-scales-only \
--tensor-type '.*=nvfp4' \
--tensor-type '^blk\..*\.attn_norm\.weight$=f32' \
--tensor-type '^blk\..*\.post_attention_norm\.weight$=f32' \
--tensor-type '^blk\..*\.attn_q_norm\.weight$=f32' \
--tensor-type '^blk\..*\.attn_k_norm\.weight$=f32' \
--tensor-type '^blk\..*\.ssm_norm\.weight$=f32' \
--tensor-type '^blk\..*\.nextn\..*norm\.weight$=f32' \
--tensor-type '^output_norm\.weight$=f32' \
--tensor-type '^blk\..*\.ssm_conv1d\.weight$=f32' \
--tensor-type '^blk\..*\.ssm_dt\.bias$=f32' \
--tensor-type '^blk\..*\.ssm_a$=f32' \
--tensor-type '^blk.0.ssm_out.weight$=nvfp4' \
--tensor-type '^blk.1.attn_gate.weight$=nvfp4' \
--tensor-type '^blk.1.attn_qkv.weight$=nvfp4' \
--tensor-type '^blk.0.attn_gate.weight$=f16' \
--tensor-type '^blk.0.attn_qkv.weight$=f16' \
--tensor-type '^blk.0.ffn_down.weight$=f16' \
--tensor-type '^blk.13.ffn_down.weight$=f16' \
Swift-Qwen3.8-27B-F16-00001-of-00003.gguf \
Swift-Qwen3.8-27B-Genesis-NVFP4-v4.gguf NVFP4 6
Credits
- Base model: ukisai/Swift-Qwen3.8-27B
- Genesis tensor repair + F16 GGUF: LuffyTheFox
- NVFP4 quantisation: jan1k
- Quantiser: advanced-gguf-quantizer