Qwen3.8-27B-Uncensored-GGUF

GGUF conversions of Qwen3.8-27B-Uncensored — an abliterated (refusal-removed) build of Qwen's Qwen3.8-27B, a 27B dense hybrid-attention (Gated DeltaNet linear + full attention) native vision-language model with reasoning, tool-calling, and an MTP speculative-decoding head. These files run in llama.cpp (CPU / CUDA / Metal / ROCm), quantized from 2-bit to 16-bit, with a separate mmproj file that restores vision. Browse all models in the OrcaRouter Model Catalog. Qwen3.8 27B is deployed as API on OrcaRouter.

⚠️ Disclaimer — read before use

This model has had its safety alignment substantially removed via abliteration (orthogonalizing the refusal direction out of the residual stream). It will comply with harmful, unethical, or illegal requests the original Qwen3.8-27B would refuse. Released strictly for legitimate research — interpretability, AI-safety / refusal-mechanism study, red-teaming, and robustness evaluation. You assume full responsibility for how you use it and everything it generates; add your own safety and moderation layers before any deployment. Use must comply with the Apache 2.0 License inherited from the base model and all applicable law. The authors accept no liability for misuse.

Requirements

• A recent llama.cpp built from source (the qwen35 hybrid-GDN architecture and the MTP / nextn speculative head — merged 2026-05 — must be present). Older releases will not load these files.
• The GDN linear-attention layers are stored as SSM-style tensors (ssm_*); full-attention layers as attn_*; the MTP head as block nextn.* (qwen35.nextn_predict_layers).

Files

Standard K-quants

IQ quants (imatrix)

Lower-bit quants built with an importance matrix (computed on English + Chinese calibration text) — better quality-per-bit than plain K-quants at the low end, especially IQ3/IQ2.

Vision

All quants (K-quant and IQ) preserve the MTP (nextn) head and the GDN hybrid architecture; vision is provided by the separate mmproj file. The IQ files were quantized with an importance matrix (computed on English + Chinese calibration text) for better low-bit fidelity; the matrix itself is not shipped, as it is only needed to re-quantize these files, not to run them.

Usage (llama.cpp)

Download

hf download orcarouter/Qwen3.8-27B-Uncensored-GGUF \
  Qwen3.8-27B-Uncensored-Q4_K_M.gguf mmproj-Qwen3.8-27B-Uncensored-f16.gguf \
  --local-dir ./qwen38-uncensored

Chat (text)

./llama-cli -m Qwen3.8-27B-Uncensored-Q4_K_M.gguf --jinja -c 8192 -p "Hello!"

OpenAI-compatible server (tool calling + reasoning + vision)

./llama-server -m Qwen3.8-27B-Uncensored-Q4_K_M.gguf \
  --mmproj mmproj-Qwen3.8-27B-Uncensored-f16.gguf \
  --host 0.0.0.0 --port 8000 -c 8192 --jinja

• Vision: pass --mmproj …, then send OpenAI image_url content parts (base64 data-URI or URL).
• Tool calling: --jinja enables the Qwen tool template; use standard OpenAI tools + tool_calls.
• Reasoning (thinking): thinking is on by default; toggle per request via chat_template_kwargs.enable_thinking. The reasoning trace is returned in reasoning_content (give max_tokens room, e.g. ≥ 2048, so the final answer isn't truncated by the thinking budget).
• MTP speculative decoding: the nextn head is embedded; enable llama.cpp's MTP/speculative path for a decode speed-up (optional — the model runs fine without it).

Or skip llama.cpp — run it with Ollama

The same quants are published as Ollama tags, with the mmproj projector already bundled in (no second file, no --mmproj flag) and vision / tools / thinking wired up:

ollama run orcarouter/Qwen3.8-27B-Uncensored          # q4_K_M by default
ollama run orcarouter/Qwen3.8-27B-Uncensored:iq4_xs   # any of the 16 tags

Sixteen tags from q2_K to q8_0, each verified on the actual quantized artifact before release — ollama.com/orcarouter.

Evaluation

Measured on this exact abliterated build (served with vLLM) vs the official Qwen/Qwen3.8-27B-FP8, using the same scripts and settings. Refusal is judged by a rule-based opening-phrase classifier — indicative, not an LLM-judge / publication-grade number. GGUF quants are deterministic derivations and inherit these behaviours; lower bits trade some quality (see the file table).

Safety — harmful-prompt refusal, thinking OFF (lower = more uncensored)

Safety — harmful-prompt refusal, thinking ON (enable_thinking=true)

Over-refusal — benign prompts wrongly refused (lower = better)

Capability retention — vs the official base FP8 (same scripts)

Harmful-prompt refusal collapses from 64–99% (base) to 0–6%; benign over-refusal drops (5.6%→0.4%); capability stays within ±1.3 pts of the base. Reasoning (enable_thinking), multi-turn tool calling (qwen3_coder), and vision (image + OCR via mmproj) all verified working on the GGUF build. Note: the above are full-precision/FP8 numbers; expect small additional degradation at lower quants (most visible at Q2_K / Q3).

Hardware

• Runs on CPU, CUDA, Metal, or ROCm via llama.cpp. VRAM/RAM ≈ the file size + KV cache + (for vision) the ~0.9 GB mmproj. E.g. Q4_K_M fits comfortably on a 24 GB GPU with room for context.

License

Apache 2.0, inherited from Qwen/Qwen3.8-27B. Abliteration and quantization do not change the underlying license obligations.