🧙‍♂️ Qwen3.8 27B Kearuga

⚡ Overview

Qwen3.8 27B Kearuga is a custom tiered sensitivity hybrid-quantized checkpoint of Qwen/Qwen3.8-27B, specifically engineered for high-throughput and interactive speculative-decoding serving on a single NVIDIA DGX Spark (GB10 / Grace-Blackwell, 128 GB Unified Memory).

By pairing GPTQ with Four-Over-Six (4o6) group scales, NVFP4 AWQ, FP8 E4M3, and protected BF16, Kearuga:

• Cuts weight footprint by 52% (from 51.8 GiB unquantized base down to 24.85 GB across 3 shards + MTP draft head).
• Preserves output fidelity to within a 0.0165 mean KL divergence against the unquantized BF16 base model, maintaining 40/40 (100%) top-1 argmax agreement.
• Unlocks fast speculative serving with SGLang: with the Kearuga DFlash 2 drafter v1.0 (K=12, NVFP4, 64K draft head) the serving repo's bench/scale.py probe measures C1 44.7 · C2 78–94 · C4 131 tok/s aggregate (T=0, thinking off, 512 forced tokens, aggregate incl. TTFT 0.23–0.34 s; identical with the GPU clock lock on or off), and the paired 50-prompt battery puts it +14.5 % C1 / +10.8 % C4 over the stock z-lab drafter (C1 net-decode medians: code 57.6, math 62.2, tool calls 114.6, prose 21.6 tok/s). Serving stack: GitHub v0.6.6; drafter release v1.0. Without speculative decoding the same target measures 10.30 tok/s C1 / 39.56 tok/s C4 on that battery (serving stack v0.6.7) — a 3.46× / 2.75× gain from the drafter.
• Runs on official SGLang Docker images — the Kearuga-drafter profile adds a 5-file read-only Python overlay (hash-verified at launch, no Docker build); the stock profile needs no overlay.

📊 Kearuga vs. Base Qwen3.8-27B (BF16)

The table below presents fidelity and quality metrics measured on an isolated NVIDIA DGX Spark (GB10 / SM121, 128 GB Unified Memory) running SGLang (KV cache in BF16, thinking disabled, temperature 0). They do not depend on the drafter — the target verifies every drafted token:

Kearuga drafter (K=12, NVFP4, 64K draft head). In a paired bake-off on the same image, target, flags and clock — 50-prompt battery (10 prompts × 5 domains) × 2 runs, temperature 0, thinking off; "net decode tok/s" = (completion tokens − 1)/(last − first token time) per request, TTFT excluded — the Kearuga DFlash 2 drafter with the draft-head overlay reached 35.33 tok/s C1 (+14.5 %) and 108.90 tok/s C4 (+10.8 %) aggregate vs the stock drafter (30.85 / 98.31), better on all 10 domain cells. C1 net-decode medians: code 46.27 → 57.62, math 51.21 → 62.18, tool calls 80.82 → 114.63, prose 19.45 → 21.64, instruction-following 19.87 → 21.80. The target still verifies every drafted token, so output is unchanged (Fidelity-40 KL 0.0165, top-1 40/40). Full tables on the drafter model card.

🛡️ Tiered Sensitivity Quantization Architecture

Uniform quantization (applying INT4 or NVFP4 indiscriminately across all layers) degrades attention sink states, vocabulary distribution tails, and speculative draft features.

Inspired by empirical mixed-precision sensitivity research (EXL3 tiered sensitivity), Kearuga partitions the model's 2,009 tensors into four distinct sensitivity tiers:

1. Four-Over-Six (4o6) Group Scaling

Standard GPTQ assigns a single static exponent code per group block (amax → code 6). Kearuga's 4o6 algorithm evaluates dynamic range block-by-block and selects the optimal scale:

• Code 6: Standard headroom for high-variance weights.
• Code 4: Higher numerical precision for weights with tight local distributions.

In our calibration, 44.7% of blocks selected Code 4, yielding a 16% reduction in Hessian-weighted MSE (0.529 → 0.445) and cutting held-out KL divergence by 21.7% without increasing tensor bit-width or memory footprint.

2. Fused-Shared Global Scale Contract

SGLang's high-performance Blackwell kernels fuse gate_proj and up_proj into a single GEMM operation, indexing a unified weight_scale_2.max(). Quantizing gate and up independently introduces scale divergence that breaks dequantization. Kearuga enforces an exact fused-shared global scale across every layer:

scale_shared = max(amax(gate), amax(up))

This ensures bit-exact kernel compatibility and eliminates dequantization distortion.

🚀 Serving & Deployment Guide

Kearuga is served directly via official SGLang Docker container images (sha256:616a3e97… or later) with native DFlash 2 speculative acceleration. The recommended drafter is 0xWhiteMage/Qwen3.8-27B-Kearuga-DFlash2 v1.0 (Kearuga-distilled, NVFP4, K=12):

docker run --gpus all \
  --shm-size 32g \
  --ulimit memlock=-1:-1 \
  --cap-add IPC_LOCK \
  -p 8888:8888 \
  -v ~/.cache/huggingface:/root/.cache/huggingface \
  lmsysorg/sglang@sha256:616a3e97f45191af975896cfa644279096cb31bd408a071c2e99ca7209c3cafe \
  python3 -m sglang.launch_server \
    --model-path 0xWhiteMage/Qwen3.8-27B-Kearuga \
    --served-model-name "Qwen3.8 27B Kearuga" \
    --speculative-algorithm DFLASH \
    --speculative-draft-model-path 0xWhiteMage/Qwen3.8-27B-Kearuga-DFlash2 \
    --speculative-draft-model-quantization modelopt_fp4 \
    --speculative-num-draft-tokens 12 \
    --speculative-draft-window-size 2048 \
    --kv-cache-dtype bf16 \
    --mem-fraction-static 0.85 \
    --max-total-tokens 1048576 \
    --context-length 262144 \
    --max-running-requests 4 \
    --host 0.0.0.0 \
    --port 8888

The 64K draft-head overlay adds a further ≈ 6 % on top of plain SGLang (35.33 vs 33.30 tok/s C1 aggregate) — it is documented on the drafter card and is handled automatically by the serving repo's launcher ./start-dflash2.sh.

Alternative: stock drafter (K=10) — the serving repo's DRAFTER_PROFILE=stock (the original v0.5.0 recipe) with z-lab/Qwen3.8-27B-DFlash2: use the command above with --speculative-draft-model-path z-lab/Qwen3.8-27B-DFlash2, no --speculative-draft-model-quantization, and --speculative-num-draft-tokens 10.

Key Serving Parameters

⏱️ Saturated Priority Scheduling

In multi-agent production pipelines, background batch traffic can cause interactive request freezes. Kearuga's serving stack enables native priority preemption:

{
  "model": "Qwen3.8 27B Kearuga",
  "priority": 100,
  "messages": [
    {"role": "user", "content": "Analyze this mission-critical trace..."}
  ]
}

🧪 Comprehensive Verification & Quality Gates

Every promoted build of Kearuga must satisfy a strict 15-gate fail-closed verification harness prior to promotion:

1. Static Tensor Census: 2,009 tensors intact across 3 shards; exactly 15 MTP tensors and 333 visual tensors confirmed byte-identical to base BF16.
2. Serving Contract Audit: Strict verification that fused gate_proj + up_proj weight_scale_2 are identical across all layers.
3. Semantic Canaries: 100% pass on exact arithmetic (19 × 23 = 437) and logic traps (9.9 vs 9.11).
4. Long-Context NIAH: 3/3 Needle-In-A-Haystack retrieval at 64K depth (25%, 50%, 90%).
5. Quality-200 Benchmark: 157/180 verified score on GSM8K, HumanEval, IFEval, and agentic coding (155/180 served with the Kearuga drafter — near-tie band 154–159).
6. Multi-Boot Stability: 3 consecutive clean cold boots with zero loader warnings or device-side asserts.

📦 Model Files & Shards

🔗 Related Resources

• Kearuga DFlash 2 Drafter v1.0: 0xWhiteMage/Qwen3.8-27B-Kearuga-DFlash2 — Kearuga-distilled, Kearuga-calibrated NVFP4 drafter + 64K draft head (served by serving stack v0.6.6)
• Stock DFlash 2 Drafter (baseline / fallback): z-lab/Qwen3.8-27B-DFlash2
• Base Model (BF16): Qwen/Qwen3.8-27B
• Creator / Maintainer: The White Mage · @0xWhiteMage on X

📜 Citation

If you use Kearuga or build upon its tiered sensitivity architecture in your work, please cite:

@misc{kearuga-2026,
  title={Kearuga: Hybrid GPTQ-4o6 + FP8 Quantization of Qwen3.8-27B for Speculative-Decoding Serve on NVIDIA DGX Spark},
  author={0xWhiteMage},
  year={2026},
  publisher={Hugging Face},
  url={https://huggingface.co/0xWhiteMage/Qwen3.8-27B-Kearuga}
}

📄 License

This checkpoint is released under the Apache 2.0 License. The underlying base model Qwen/Qwen3.8-27B is subject to its original license terms from Alibaba Cloud.