Qwen3.8-27B (GGUF target for 16GB VRAM)

• This repository provides GGUF quantizations for Qwen3.8-27B optimized using ZB-ZipBrain, a custom mixed-precision quantization methodology that optimizes LLM tensor bit allocation using rate-distortion marginal cost combined with importance matrix calibration. It automatically identifies Pareto-optimal BPW "sweet spots" to maximize model quality while fitting precise VRAM and memory footprints.
• Specifically optimized to fit mainstream GPUs within a 16GB VRAM budget at around 4 BPW, and even 12GB VRAM cards at around 3 BPW.
• For filenames marked with v5, I combined ZB + Pelicanmaxxing for visual evaluation, iteratively tuning until the output reached the most stable quality before locking it in.

Benchmark & Evaluation Results

EvalPlus Benchmark Results

• HumanEval: 164 tasks
• MBPP: 378 tasks
• Scores are pass@1, no-thinking mode , kvcache ctk q4_0, ctv q4_0
• “—” indicates data not provided.

Comprehensive Comparison Table

Command llama-perplexity.exe -f /wikitext-2-raw/wiki.test.raw --kl-divergence --kl-divergence-base q38f16baseline.kld -ngl 99 -m model.gguf

All models were evaluated against the BF16 baseline (Mean PPL = 6.950493) using standard Perplexity (PPL) and KL Divergence metrics.

Update: Aug 20, 2026

• The newly released Unsloth Dynamic v3 is truly the best value for performance right now.
• My ZB is just an experiment, feel free to check it out for fun :)

Update: Aug 22, 2026

• Quant release ZB-4.00 BPW runs cleanly on 16GB VRAM with MTP support and up to 95K context length.

Update: Aug 24, 2026

• ZBv3-4.00BPW update maintaining size with KLD and Same top-p performs slightly better. ZBv2-4.00BPW -> ZBv3-4.00BPW: output weights were bumped from Q5_K to Q6_K. Testing shows sharper, more consistent outputs and better one-shot performance. => Go with ZBv3-4.00BPW.
• ZB3.7-MIN ⚔️ empero-ai/Qwen3.8-27B-Ridge. 🤣
• I have just updated empero-ai/Qwen3.8-27B-Ridge. benchmarks; comparing my ZB 3.7bpw metrics, it looks like it easily beats down Qwen3.8-27B-Ridge

Update: Aug 26, 2026

• ZB3.7-MIN-v4 update PPL slightly better

Update: Aug 28, 2026

• All use Q6_K for output.weight
• Please prioritize later development versions, I have removed older ones because I was not satisfied with them.
• Quant release 4.00bpw & 3.88bpw v5 (v5 = ZB + Pelicanmaxxing and visually check for other aspects of stability. )
• ZB3.88-MIN-v5-IQ3_M_XL ⚔️ UD3-Q3_K_XL 😎 If anyone has used this pair, please let me know what you think.
• 3.0BPW-IQ3_XXS: New release size only 9.62 GB 🙀 with metric comparable to Ridge—3.7 bpw.

Update: Sep 5, 2026

• ZB4.00-MIN-v5.1-IQ4_XS This version has been updated so that all tensors are ≥ IQ3_XXS , previous version contained some IQ2_S tensors. Quality is slightly improved, new file size saves 50MB.

Update: Sep 8, 2026

• ZB3.73-MIN-v5.1-IQ3_M_L Added new 3.73 bpw. Updated all tensors ≥ IQ3_XXS, quality is slightly improved. Change for ZB3.70-MIN-v4-IQ3_M_L.
• Added HumanEval, MBPP benchmark

Recommended Settings: Set reasoning_effort to medium. At this BPW level, it delivers much more stable outputs and fits well in agentic workflows. You can also use the default settings for higher quality, though it will take longer.

llama-server -m models/qwen38/Qwen3.8-27B-ZB4.00-MIN-IQ4_XS.gguf -mm models/qwen38/Qwen3.8-27B-mmproj-BF16.gguf --host 0.0.0.0 --port 8080 --temp 1 --top-p 0.95 --top-k 20 --min-p 0.00 --reasoning-preserve -ctk q4_0 -ctv q4_0 -fa on --ubatch-size 384 --batch-size 384 --no-mmproj-offload --spec-type draft-mtp,ngram-mod --spec-draft-n-max 2 --spec-ngram-mod-n-match 24 --spec-ngram-mod-n-min 24 --spec-ngram-mod-n-max 32 -ngl 99 -t 7 --ctx-size 95000 -np 1 --load-mode mlock --image-min-tokens 1024 --image-max-tokens 2048 --chat-template-kwargs '{\"reasoning_effort\": \"medium\"}'

ZB Tiers & Recommendations

• ZB-GOD : God. A singularity appears. Reaches 0.012249 Mean KLD and 95.34% top-probability match.
• ZB-PRO : Pro. For 16 GB VRAM GPUs with offloading on CPU. Balances quality output with substantial size savings.
• ZB-STD : Standard. Similar to other standard IQ4_XS models currently available.
• ZB-MIN : Minimal. Optimal footprint for tight 16 GB memory setups, allowing headroom for longer context windows

Credits & Acknowledgements

• Base Model: Qwen3.8 27B by Alibaba Cloud / Qwen Team.
• BF16 Base GGUF: Provided by Unsloth AI.
• Importance Matrix (imatrix): Generated and curated by ubergarm.
• Inference & Quantization Framework: llama.cpp by Georgi Gerganov and contributors.