Qwen3.6-35B-A3B-Uncensored-Genesis — NVFP4 GGUF
NVFP4 quantisation of LuffyTheFox/Qwen3.6-35B-A3B-Uncensored-Genesis-Final-GGUF, built with advanced-gguf-quantizer (a llama.cpp fork focused on NVFP4/MXFP6 quantization).
This is the non-Hermes version — Genesis tensor repair on the HauhauCS uncensored base, without the Hermes finetune transfer. For the Hermes version, see jan1k/Qwen3.6-35B-A3B-Uncensored-Genesis-Hermes-Final-NVFP4-GGUF.
Files
v4 — Recommended (inline scales, LM Studio compatible)
| File | Size | Tensors |
|---|---|---|
Qwen3.6-35B-A3B-Uncensored-Genesis-NVFP4-v4.gguf | ~20 GB | 733 |
mmproj-Qwen3.6-35B-A3B-Uncensored-Genesis-Final-F16.gguf | ~899 MB | vision projector |
Re-quantised 2026-09-24 from Luffy's updated Q8_K_P source.
v4 uses --nvfp4-inline-scales-only to emit native inline UE4M3 scales
without separate .scale/.input_scale tensors. This is required for
LM Studio / Pelican and other runtimes that do not support the extended
NVFP4 scale tensor contract.
Imatrix variants will follow after the plain v4 is confirmed working in target runtimes.
Source
| Source GGUF | Qwen3.6-35B-A3B-Uncensored-Genesis-Final-Q8_K_P.gguf (43.6 GB, 10.06 BPW) |
| Architecture | qwen35moe (MoE), 35B total / ~3B active, 40 layers |
| Context | 262144 |
general.file_type | 39 (LLAMA_FTYPE_MOSTLY_NVFP4) |
| MTP/NextN | none |
Two-step pipeline: Q8_K_P -> F16 intermediate -> NVFP4. The F16 step gives the NVFP4 encoder clean data (no Q8_0 quantization noise).
Tensor mix (v4)
| type | count | notes |
|---|---|---|
| F32 | 331 | norms, ssm scalars, gate inputs |
| F16 | 175 | sensitive weights (blk.0 attn, ssm, etc.) |
| Q6_K | 1 | output.weight |
| NVFP4 | 226 | bulk weights |
| total | 733 | |
separate .scale/.input_scale | 0 | inline UE4M3 only |
Format fix (v3/v2 -> v4)
The older v2/v3 files use the extended NVFP4 format with separate
.scale and .input_scale tensors. Runtimes like LM Studio / Pelican
only read inline UE4M3 scales and ignore the separate scale tensors,
producing garbage output.
v4 uses --nvfp4-inline-scales-only to:
- Skip writing
.scaleand.input_scaletensors - Force
tensor_scale=1.0(no pre-scaling) - Force
input_scale=identity - Make inline UE4M3 scales self-contained
Tensor protection policy
F16 singular-collapse protection:
| tensor | type |
|---|---|
blk.0.attn_gate.weight | F16 |
blk.0.attn_qkv.weight | F16 |
blk.0.ffn_down_exps.weight | F16 |
blk.13.ffn_down_exps.weight | F16 |
F32 architecture-specific protection:
| tensor | reason |
|---|---|
blk.*.attn_norm.weight | 1D norm |
blk.*.post_attention_norm.weight | 1D norm |
blk.*.attn_q_norm.weight | 1D norm |
blk.*.attn_k_norm.weight | 1D norm |
blk.*.ssm_norm.weight | 1D norm |
output_norm.weight | 1D norm |
blk.*.ssm_conv1d.weight | CUDA SSM conv kernel requires F32 |
blk.*.ssm_dt.bias | SSM scalar, kernel compatibility |
blk.*.ssm_a | SSM scalar, kernel compatibility |
blk.*.ffn_gate_inp_shexp.weight | shared expert gate input |
token_embd.weight | F16 (embedding quality) |
Forced NVFP4 (do not push lower):
| tensor | type |
|---|---|
blk.0.ssm_out.weight | NVFP4 |
blk.1.attn_gate.weight | NVFP4 |
blk.1.attn_qkv.weight | NVFP4 |
Usage
llama-cli -m Qwen3.6-35B-A3B-Uncensored-Genesis-NVFP4-v4.gguf \
--mmproj mmproj-Qwen3.6-35B-A3B-Uncensored-Genesis-Final-F16.gguf \
--jinja -c 131072 -ngl 99
- Set K cache and V cache quantization to F16
- Set GPU offload to maximum, active experts to 8
- Set number of layers for which to force MoE weights onto CPU to 40
Hardware
- Blackwell (RTX 50xx): native FP4 path, fastest
- Ampere (RTX 30xx): NVFP4 inference works via fallback kernels
- Quantisation was done CPU-only (Ampere CUDA NVFP4 encoder hangs on MoE)
Reproducibility
# Step 1: Q8_K_P -> F16 intermediate
llama-quantize --allow-requantize \
Qwen3.6-35B-A3B-Uncensored-Genesis-Q8_K_P.gguf temp_f16.gguf F16 6
# Step 2: F16 -> NVFP4 v4 (inline scales only, CPU-only)
CUDA_VISIBLE_DEVICES="" llama-quantize \
--allow-requantize \
--nvfp4-inline-scales-only \
--tensor-type-file tensor_types_protection.txt \
temp_f16.gguf \
Qwen3.6-35B-A3B-Uncensored-Genesis-NVFP4-v4.gguf \
NVFP4 6
# Cleanup
rm temp_f16.gguf
Credits
- Base model: HauhauCS (0/465 refusals)
- Genesis tensor repair: LuffyTheFox
- NVFP4 quantisation: jan1k
- Quantiser: advanced-gguf-quantizer