Qwen3.8-27B Uncensored · W4A16 + MTP4
A complete uncensored checkpoint optimized for speculative decoding on a single 24 GB GPU. 100K text context with BF16 KV cache and MTP4 is tested on an RTX 3090 Ti.
The W4A16 transformer body is paired with INT8 token embeddings, calibrated GPTQ INT4 output/MTP heads, and a 40,960-token draft vocabulary selected from this model's own outputs. Group size is 128. The vision tower is retained; the 100K serving profile is text-only.
100K serving setup
Use the patched syv-ai runtime, tested with vLLM 0.27.1. 100K profile details.
Download into the runtime repository:
hf download protogeni/Qwen3.8-27B-Uncensored-W4A16-RTX3090-MTP4 \
--local-dir models/Qwen3.8-27B-Uncensored-W4A16-RTX3090-MTP4
Merge these settings into its .env:
MODEL=/app/models/Qwen3.8-27B-Uncensored-W4A16-RTX3090-MTP4
CTX=fast
SPEC=mtp
DRAFT_TOKENS=4
PREFIX_CACHE=1
VISION=0
GPU_UTIL=0.95
MAX_LEN=100000
MAX_SEQS=4
CG=8
EXTRA_ARGS=--kv-cache-memory=7730941132 --max-num-batched-tokens 512
docker compose --profile single up -d --no-deps --force-recreate single
Set the client's context limit to 100,000 tokens, including the reply.
Measured performance
RTX 3090 Ti, BF16 KV + MTP4. Generation rates exclude prompt processing and include reasoning tokens.
| Workload | Generation speed |
|---|---|
| Three short coding prompts, pooled | 128.4 tokens/s |
| 64,230 input + 1,024 output tokens | 92.1 tokens/s |
| 96,238 input + 3,072 output tokens | 82.7 tokens/s |
| 98,233 input + 1,536 output tokens | 84.6 tokens/s |
Quality results
| Benchmark | Result |
|---|---|
| GSM8K | 94.77% — 1,250/1,319 |
| MMLU-Pro, 5-shot | 76.14% — 1,066/1,400 |
| HumanEval | 94.51% pass@1 — 155/164 |
| HumanEval+ | 90.85% pass@1 — 149/164 |
| IFBench | 39.33% strict / 42.00% loose |
| MMMU validation | 24.56% — 221/900; 891 output-cap hits |
MMLU-Pro used thinking; the other listed tests used thinking off. MMMU used an eager, MTP-off profile. Protocols, raw results, and earlier speed tests.
Lineage and license
Qwen/Qwen3.8-27B → OrcaRouter uncensored → noon-at-cgn AutoRound W4A16 → this serving-optimized checkpoint.