⚠️ Known issue (2026-08-27): generation can degenerate into loops under some conditions — investigating. Not recommended for deployment yet. For a validated uncensored Qwen3.8-27B NVFP4 serving path, see the blog post covering the current working alternative.
Qwen3.8-27B-OBLITERATED-NVFP4
NVFP4 (W4A16) quantization of OBLITERATUS/Qwen3.8-27B-OBLITERATED (V3 abliteration, zero refusals and zero deflection lectures, -2.1pp MMLU vs stock), built with NVIDIA ModelOpt 0.45.0 and packaged for vLLM with the MTP head preserved and wired for speculative decoding.
21.5 GB on disk versus 55.6 GB for the BF16 source. Built for and validated on a single RTX 5090 (32 GB, SM120); the same layout loads on any Blackwell card and on FP4-capable datacenter GPUs.
What is in the box
model.safetensors(19.7 GB): FP4-packed weights, deployment tensor naming (weight_scale/weight_scale_2), vision tower in BF16model_mtp.safetensors(1.7 GB): the 15 BF16 MTP tensors, mapped through the safetensors index (vLLM convention)- Flat ModelOpt
quantization_configinconfig.json(compressed-tensorsconfig_groups+quant_algo: W4A16_NVFP4+quant_method: modelopt) - Tokenizer, chat template, and processor configs unchanged from the upstream release
(tool-call dialect is
qwen3_coder)
Excluded from quantization (kept BF16): vision tower, MTP head, lm_head, embeddings,
and the conv1d / in_proj_a / in_proj_b internals of the linear-attention layers.
The fused in_proj_qkv / in_proj_z projections are intentionally quantized: on
this architecture they must ship packed or vLLM's fused-module loader rejects them.
Serving with vLLM
vllm serve /path/to/model \
--served-model-name qwen3.8-27b-obl \
--max-model-len 126976 \
--gpu-memory-utilization 0.94 \
--kv-cache-dtype fp8_e4m3 \
--max-num-seqs 8 \
--speculative-config '{"method":"qwen3_5_mtp","num_speculative_tokens":4}' \
--mamba-cache-mode align \
--reasoning-parser qwen3 \
--enable-auto-tool-choice --tool-call-parser qwen3_coder
Notes:
- The MTP head was restored from stock weights by the OBLITERATUS authors, so it drafts
for abliterated weights it never saw: expect lower acceptance than stock. If decode
regresses, drop
--speculative-configand keep the concurrency. qwen3_coderis the correct tool parser for this checkpoint's template (<tool_call><function=NAME>), notqwen3_xml.- Vision input works; the tower is untouched BF16.
Measured on RTX 5090 (32 GB), vLLM, 126K context, FP8 KV
| Load | Throughput |
|---|---|
| Single stream, MTP ns=4 | ~40 tok/s |
| 4 concurrent streams | 161 tok/s aggregate |
| 8 concurrent streams | 317 tok/s aggregate (~40 per stream) |
| GPU KV cache | 191,118 tokens |
| Warm startup | ~140 s |
Intended use and limitations
This is an abliterated ("uncensored") derivative: refusal behavior has been removed upstream. It will comply with requests the base model would decline. You are responsible for the legality and consequences of what you generate, and for adding whatever guardrails your deployment requires. Do not expose it unauthenticated to the public internet.
Quantization was not calibrated per task; validate against the BF16 upstream for task-critical accuracy. The base model's knowledge, biases, and failure modes carry over.
Lineage
Qwen/Qwen3.8-27B (Apache-2.0) -> OBLITERATUS V3 abliteration -> ModelOpt 0.45.0
W4A16 NVFP4 (this repo). Full build story and pitfall list:
Three Days, Four Wrong Hypotheses, and One Uncensored 27B That Finally Serves on a Single RTX 5090
Thanks to OBLITERATUS for the careful abliteration work, to hwkranger's Qwen3.8-27B-heretic-ara-NVFP4 whose public tensor layout was the blueprint for a loadable checkpoint, and to the Unsloth team for pushing NVFP4 on consumer Blackwell.