SuperQwen3.8-27b-abliterated-GGUF

A one-DGX-Spark Q4_K_M release with a native MTP draft, Q8_0 vision projector, bounded reasoning, and a verified 262K context path.

This is the llama.cpp build of Jiunsong/SuperQwen3.8-27b-abliterated: the corrected rank-4 OBLITERATUS model, not an adapter and not the earlier incorrect lineage. The target, MTP draft, and multimodal projector are separate so text-only users do not pay for the projector, while vision and native speculative decoding remain available.

Release highlights

Why this quant

• Built for one DGX Spark: the whole runtime set is 17.56 GiB.
• Speed selected by measurement: every K from 0 through 8 was tested under the same single-stream fixed-generation contract; K=6 was then reloaded independently.
• Multimodal remains real: the projector is shipped and the post-quant vision gate passed.
• Behavior survived compression: capability 8/8, tool use, bounded reasoning, refusal, and native-context retrieval all passed on this GGUF itself.

Files and integrity

The target contains 433 Q4_K, 65 Q6_K, and 353 F32 tensors. The MTP draft contains 10 Q4_0 and 8 F32 tensors. The projector deliberately retains unsupported or sensitive matrices: 83 Q8_0, 27 F16, and 224 F32 tensors.

Target quantization used the Unsloth Qwen3.8 importance matrix with SHA-256 0ee5b10bd0c2fa2127c6f4b43dbfe1efd71e383b63217af9dade1de36599f1c1. Conversion and serving are pinned to llama.cpp b3c3b96a139d4ef1bdec926ac17aa040981cfc5d.

Native MTP speculation

The selected K=6 profile produced independent C1 trials of 40.100, 40.017, 40.446 tok/s. Its final median is 40.100 tok/s, versus 12.052 tok/s for K=0 in the complete scan (3.33×). C1 means one active request; it is not a concurrency aggregate relabeled as interactive speed.

Serving on one DGX Spark

llama-server \
  -m SuperQwen3.8-27b-abliterated-Q4_K_M.gguf \
  -md mtp-SuperQwen3.8-27b-abliterated-Q4_0.gguf \
  --spec-type draft-mtp \
  --spec-draft-n-max 6 \
  --spec-draft-ngl 999 \
  --spec-draft-type-k q8_0 \
  --spec-draft-type-v q8_0 \
  -mm mmproj-SuperQwen3.8-27b-abliterated-Q8_0.gguf \
  -ngl 999 -c 262144 -ctk q8_0 -ctv q8_0 -fa on -np 1 \
  --host 0.0.0.0 --port 8891 \
  -a SuperQwen3.8-27b-abliterated-GGUF --jinja --reasoning-format deepseek

For a non-speculative baseline, omit -md and use --spec-type none. For text-only serving, omit -mm. Reduce -c when you prefer a smaller KV allocation.

Independent release gates

Verified long context

The selected K=6 server processed a 262,043-token prompt and returned the hidden retrieval key. This is a concrete retrieval gate, not a claim that every task at the edge of the window will be equally accurate.

Other formats

• Full BF16
• Single-DGX NVFP4 W4A4
• Apple Silicon MLX 4-bit

Uncensored behavior

“Abliterated” means a measured refusal subspace was reduced. It does not mean every answer is correct, harmless, or appropriate for every deployment. Operators remain responsible for access controls and downstream safeguards.

Limitations

• Q4_K_M can regress tasks outside the measured gates; use BF16 for maximum fidelity.
• MTP speedup depends on prompt distribution and acceptance; remeasure your workload.
• Context, speed, and memory figures are specific to the pinned llama.cpp runtime and GB10.
• The evaluation suites are strong regression gates, not exhaustive guarantees.

Evidence identities

SHA256SUMS.json covers every published file. The repository remains private until all BF16, NVFP4, GGUF, and MLX artifacts pass exact remote SHA-256 verification together.

License

Apache-2.0, following the upstream Qwen3.8 release.