Qwen3.8-27B-FP8-Pessoa

FP8 (W8A16) weight-only, GPTQ-quantized variant of Qwen/Qwen3.8-27B, produced with llm-compressor and saved in compressed-tensors format for direct use with vLLM.

Pessoa is the best FP8 quantization of Qwen3.8-27B of a set of experiments with autoresearch it reached the the lowest KLD (0.0056) achieved by any practical-size FP8 model in a 97-model quantization sweep. Every linear language-model layer MLP, self-attention, and linear-attention uses FP8 E4M3 weight-only (W8A16) with per-channel weights, BF16 activations, and GPTQ Hessian error correction. The vision tower and MTP head are kept in BF16, so the model retains full image/video understanding and multi-token-prediction speculative decoding capability.

Named after Fernando Pessoa (1888–1935), the Portuguese poet who wrote under dozens of heteronyms — each a distinct literary persona with its own voice. This model too has multiple selves: the quantized language model, the unquantized vision tower, and the MTP speculative head three complete personas in one checkpoint.

Model Details

Qwen3.8-27B (qwen3_5 / Qwen3_5ForConditionalGeneration):

• Type: Vision-language model (hybrid linear + full attention language model + vision encoder + MTP head)
• Parameters: 27B (language model) + 460.7M (vision tower) + 82.3M (MTP head)
• Layers: 64 — 48 linear-attention (GatedDeltaNet) + 16 full-attention (every 4th layer: 0, 4, 8, …, 60)
• Hidden size: 5,120
• Intermediate size: 17,408 (MLP is 3.4× hidden — dominates parameter count)
• Attention heads: 24 × 256 dim
• KV heads: 4 (GQA 6:1) — only on 16 full-attention layers
• Vocabulary: 248,320
• Activation: SiLU
• Context: 262,144 tokens native
• Vision encoder: 27-layer ViT, hidden size 1,152, patch size 16, 16 heads, temporal patch size 2
• MTP: 1 hidden layer, multi-token-prediction speculative decoding head

Each linear-attention layer has in_proj_qkv, in_proj_z, out_proj plus recurrent state paths in_proj_a/in_proj_b. Each full-attention layer has standard q/k/v/o_proj. All layers share gate_proj/up_proj/down_proj MLP blocks. The vision tower (model.visual.*) processes image/video patches and injects embeddings into the language model via image/video token boundaries. The MTP head (mtp.*) provides speculative decoding tokens for inference acceleration.

Quantization Configuration

Precision Assignment

Key difference from W8A8: This model uses weight-only FP8 (W8A16) — weights are FP8 E4M3 per-channel, but activations remain BF16. This eliminates activation quantization error entirely, halving KLD versus W8A8 (0.0056 vs 0.0102). The tradeoff is ~18% slower prefill (4,875 vs 6,213 tok/s) because BF16 activations require dequantization before GEMM.

GPTQ: Hessian-based weight correction, sequential targets Qwen3_5DecoderLayer, actorder=static, dampening_frac=0.01, block_size=128. GPTQ corrects the weight quantization residual by adjusting remaining weights using the calibration data's second-order statistics — this is what gives Pessoa 2.3× lower KLD than ModelOpt's max-calibration FP8 (0.0056 vs 0.0127).

Tensor Inventory

Modules Kept in BF16

Calibration

• Source: malaiwah/qwen38-27b-fidelity-suite-v3 tokens (181 contexts × 2048 tokens)
• Samples: 181
• Max sequence length: 2048
• Format: Decoded token IDs from suite, not raw text
• Dampening: dampening_frac=0.01 (default — confirmed optimal for FP8; tested 0.001–0.1, all within noise)

Compressed-Tensors Config

{
  "quant_method": "compressed-tensors",
  "format": "float-quantized",
  "quantization_status": "compressed",
  "version": "0.18.0",
  "config_groups": {
    "group_0": {
      "targets": ["Linear"],
      "weights": {
        "num_bits": 8,
        "type": "float",
        "symmetric": true,
        "group_size": null,
        "strategy": "channel",
        "dynamic": false,
        "actorder": "static",
        "observer": "memoryless_minmax"
      },
      "input_activations": null
    }
  },
  "kv_cache_scheme": {
    "num_bits": 8,
    "type": "float",
    "symmetric": true,
    "strategy": "tensor",
    "dynamic": false,
    "observer": "static_minmax"
  }
}

Evaluation

Distribution Fidelity (KLD)

Hidden-state capture after final RMSNorm → shared BF16 LM-head replay → full-vocabulary KL(BF16_ref ‖ candidate), two-pass log-sum-exp normalization. Body-only comparison (candidate LM head unused). Suite: 136 contexts × 2048 tokens, 278,392 scored positions over 248,320-token vocabulary.

KLD by Domain

Literary text remains the dominant KLD contributor — 5.1× the scientific stratum. Creative/prose generation is disproportionately affected by quantization, but the gap is narrower than in W8A8 configs (v17: 5.4×) and much narrower than 4-bit configs. Pessoa's weight-only approach preserves activation distributions, keeping literary KLD under 0.01.

Throughput

Single NVIDIA RTX PRO 6000 Blackwell (SM120, 96 GB), vLLM 0.26.1rc0, FlashInfer backend. Duration-based sustained decode, 30s cells, temperature 0.0, ignore_eos=true.

NOTE: This was not optimized for speed, but for equal grounds between all experiments and models tested. An optimized setup with a 6kpro reaches >100tok/s with MTP.

Composite Score

$$\text{score} = \frac{\text{prefill}{2k} \times \text{decode}{c1}}{\text{KLD} \times 10^6} = \frac{4875 \times 45.6}{0.00556 \times 10^6} = \mathbf{40.02}$$

Highest composite score among all practical-size FP8 models in the 97-model sweep.

Context Within Experiment Series

Pessoa is the FP8 weight-only winner from a systematic 97-model quantization sweep of Qwen3.8-27B, exploring NVFP4, MXFP8, and FP8 formats via both ModelOpt (NVIDIA) and GPTQ (llm-compressor). The sweep evaluated GPTQ vs ModelOpt, weight-only vs weight+activation, single-format vs mixed-precision, and per-layer sensitivity.

Key Finding: Weight-Only Beats Weight+Activation for FP8

Weight-only FP8 eliminates activation quantization error entirely. GPTQ corrects the weight quantization residual. The result is 2× better KLD than W8A8 and 2.3× better than ModelOpt at the cost of ~18% slower prefill (BF16 dequantization before GEMM).

Key Finding: GPTQ Beats ModelOpt by 2.3×

GPTQ's Hessian-based weight correction compensates for quantization error by adjusting remaining weights using calibration data's second-order statistics. ModelOpt's max calibration only computes scale factors without weight correction.

Pareto Position

Pessoa dominates the practical-size Pareto frontier: 2× lower KLD than any other ~28 GB model, with the highest composite score in its class.

Usage

# vLLM serve (requires Blackwell SM120 + FlashInfer)
vllm serve /models/Qwen3.8-27B-FP8-Pessoa \
  --max-model-len 262144 \
  --max-num-seqs 16 \
  --quantization compressed-tensors \
  --kv-cache-dtype fp8 \
  --trust-remote-code \
  --speculative-config '{"method":"mtp","num_speculative_tokens":3}'

--max-num-seqs matters on this architecture. 48 of 64 layers use linear attention, and vLLM allocates one Mamba-style cache block per decode sequence. The default max_num_seqs=1024 can exceed available blocks and fail during CUDA graph capture with max_num_seqs (1024) exceeds available Mamba cache blocks. Lower to 16–512 or raise --gpu-memory-utilization. This is a property of the base model, not of quantization.

from vllm import LLM, SamplingParams

llm = LLM(model="/models/Qwen3.8-27B-FP8-Pessoa",
          quantization="compressed-tensors",
          kv_cache_dtype="fp8",
          max_model_len=262144,
          trust_remote_code=True)

# Text-only generation
out = llm.generate(
    ["Explain 4-bit quantization in two sentences."],
    SamplingParams(temperature=0.7, max_tokens=256),
)
print(out[0].outputs[0].text)

# Vision: image + text via chat template
from vllm.inputs import TextPrompt
messages = [{
    "role": "user",
    "content": [
        {"type": "image_url", "image_url": {"url": "https://example.com/image.png"}},
        {"type": "text", "text": "Describe this image."},
    ],
}]
prompt = llm.get_tokenizer().apply_chat_template(
    messages, tokenize=False, add_generation_prompt=True)
out = llm.generate(
    {"prompt": prompt, "multi_modal_data": {"image": image_placeholder}},
    SamplingParams(temperature=0.7, max_tokens=512),
)
print(out[0].outputs[0].text)

Docker Compose

A docker-compose.yml is provided in the experiment workspace for one-command deployment. The image bundles the vLLM infernal-invocation fork with FlashInfer and SM120 kernels; only the quantized checkpoint must be mounted from the host.

# Place the checkpoint at ./models/Qwen3.8-27B-FP8-Pessoa, then:
docker compose up -d

# Or override the model path and port:
MODEL_DIR=/data/models/Qwen3.8-27B-FP8-Pessoa PORT=8001 docker compose up -d

The endpoint is OpenAI-compatible: http://localhost:8000/v1.

# docker-compose.yml — Qwen3.8-27B-FP8-Pessoa
#
# Serves the FP8 W8A16 weight-only GPTQ quantization via mainline vLLM.
# Image: vllm/vllm-openai:latest
#
# Requirements:
#   - NVIDIA Blackwell (SM120) GPU with FP8 E4M3 tensor cores
#   - NVIDIA Container Toolkit (nvidia-docker) installed on host
#   - Quantized checkpoint at ${MODEL_DIR}
#
# Customize via .env file or shell environment variables:
#
#   PORT=8000                           API server port
#   SERVED_MODEL_NAME=qwen38-27b-pessoa Model name returned by /v1/models
#   MAX_MODEL_LEN=262144                Maximum context length (tokens)
#   MAX_NUM_SEQS=16                     Maximum concurrent decode sequences
#   MAX_NUM_BATCHED_TOKENS=16384        Maximum tokens per batching iteration
#   GPU_MEMORY_UTILIZATION=0.97         Fraction of GPU memory for vLLM (0.0–1.0)
#   TENSOR_PARALLEL_SIZE=2              Tensor-parallel degree (1, 2, or 4)
#   KV_CACHE_DTYPE=fp8                  KV cache dtype: fp8 | auto | bfloat16
#   CUDA_VISIBLE_DEVICES=0,1            GPU indices to use
#   MODEL_DIR=./models/Qwen3.8-27B-FP8-Pessoa  Host path to checkpoint

services:
  qwen38-27b-pessoa:
    image: vllm/vllm-openai:latest
    container_name: qwen38-27b-pessoa
    ipc: host
    ports:
      - "${PORT:-8000}:${PORT:-8000}"
    volumes:
      - ${MODEL_DIR:-./models/Qwen3.8-27B-FP8-Pessoa}:/models/Qwen3.8-27B-FP8-Pessoa:ro
      - vllm-cache:/cache
      - hf-cache:/data/hf_cache
    environment:
      - CUDA_VISIBLE_DEVICES=${CUDA_VISIBLE_DEVICES:-0}
      - PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True
      - OMP_NUM_THREADS=16
      - HF_HOME=/data/hf_cache
      - VLLM_CACHE_ROOT=/cache/vllm
      - TORCHINDUCTOR_CACHE_DIR=/cache/torchinductor
      - TRITON_CACHE_DIR=/cache/triton
    entrypoint: ["python3", "-m", "vllm.entrypoints.cli.main"]
    command:
      - serve
      - /models/Qwen3.8-27B-FP8-Pessoa
      - --served-model-name=${SERVED_MODEL_NAME:-qwen38-27b-pessoa}
      - --host=0.0.0.0
      - --port=${PORT:-8000}
      - --trust-remote-code
      - --quantization=compressed-tensors
      - --tensor-parallel-size=${TENSOR_PARALLEL_SIZE:-1}
      - --disable-custom-all-reduce
      - --kv-cache-dtype=${KV_CACHE_DTYPE:-fp8}
      - --block-size=128
      - --gpu-memory-utilization=${GPU_MEMORY_UTILIZATION:-0.96}
      - --max-model-len=${MAX_MODEL_LEN:-262144}
      - --max-num-seqs=${MAX_NUM_SEQS:-16}
      - --max-num-batched-tokens=${MAX_NUM_BATCHED_TOKENS:-16384}
      - --max-cudagraph-capture-size=64
      - --generation-config=vllm
      - --reasoning-parser=qwen3
      - --attention-backend=flashinfer
      - --enable-chunked-prefill
      - --enable-prefix-caching
      - --enable-auto-tool-choice
      - --tool-call-parser=qwen3_coder
      - --mm-encoder-tp-mode=data
      - --speculative-config={"method":"mtp","num_speculative_tokens":3}
      - >-
        --compilation-config={"cudagraph_mode":"FULL_AND_PIECEWISE","custom_ops":["all"]}
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              capabilities: [gpu]

volumes:
  vllm-cache:
  hf-cache:

Configurable variables (set in .env or shell):

Provenance & Integrity

The quantized LM weights (shards 1–2) are byte-identical to the original text-only gptq-fp8-w8a16-mlp checkpoint. Shard 3 contains 333 BF16 visual + 15 BF16 MTP tensors extracted from the base model — no re-quantization was performed on these.

Hardware & Runtime

Reproduction

Step 1: FP8 W8A16 weight-only GPTQ quantization

# Inside Docker (vLLM infernal-invocation image, llm-compressor, SM120)
CUDA_VISIBLE_DEVICES=0 SUITE_DIR=/data/suite-v3 \
  python scripts/quant_gptq_unified.py \
  --model /data/models/Qwen3.8-27B-bf16 \
  --output /data/models/Qwen3.8-27B-gptq-fp8-w8a16-mlp \
  --format fp8-w8a16-mlp --calib-samples 181

Calibration tokens from $SUITE_DIR/tokens (default /data/suite-v3/tokens). Script: scripts/quant_gptq_unified.py. Format: fp8-w8a16-mlp (FP8 E4M3 per-channel weight-only for all Linear, BF16 activations, GPTQ Hessian correction).

Step 2: Merge visual tower + MTP from base

# Merge quantized LM shards with BF16 visual+mtp from base model
python /workspace/scripts/merge_vision_mixed.py \
  --src /models/Qwen3.8-27B-gptq-fp8-w8a16-mlp \
  --base /models/Qwen3.8-27B-bf16 \
  --output /data/models/Qwen3.8-27B-FP8-Pessoa

Script: scripts/merge_vision_mixed.py. Extracts 333 visual + 15 MTP tensors from the BF16 base, writes them as a 3rd shard, copies the 2 quantized shards unchanged, and patches config.json to Qwen3_5ForConditionalGeneration (VLM) with vision_config, text_config, and re:^mtp.* / re:.*visual.* added to the quantization ignore list.

Limitations

• Weight-only means slower prefill. BF16 activations require dequantization before GEMM, making prefill ~18% slower than W8A8 (4,875 vs 6,213 tok/s). Decode speed is comparable (45.6 vs 46.6 tok/s). If prefill throughput matters more than KLD, use the W8A8 variant (v17).
• Blackwell-optimized. FP8 E4M3 MMA is supported on SM120 (Blackwell), SM89/SM90 (Ada/Hopper), but the benchmark and KLD capture were run on SM120 only. Validate on your hardware.
• Body-only KLD. Fidelity measured against a shared BF16 LM head; end-to-end generation quality may differ slightly.
• Literary domain degradation. 5.1× higher KLD on literary text vs scientific — creative/prose generation is disproportionately affected, though less severely than in W8A8 or 4-bit configs.
• MTP speculative decoding acceptance depends on vLLM version and configuration. Ensure re:^mtp.* is in the quantization ignore list (it is, in this checkpoint) so vLLM's compressed-tensors loader leaves the BF16 MTP weights untouched.
• Quantization is lossy. Validate on your own workload before production use.

Acknowledgements

Standing on the shoulders of giants. This work would not exist without the rtx6kpro community and the broader local-inference-lab Discord — the open protocol, tooling, and baseline measurements that made systematic quantization comparison possible.

The distribution-fidelity (KLD) methodology follows the published protocol from Kimi-K3 distribution fidelity 1024×2048. The reference harness (glm52_exl3_shared_h_kld.py) and the benchmark tool (llm-inference-bench) are adapted from the rtx6kpro repository. The Gilded Gnosis EXL3 model cards established the per-bit error-ladder and validation-tier framework that this sweep builds on.

"I am the empty stage where the actors perform their various roles." — Bernardo Soares (heteronym of Fernando Pessoa), The Book of Disquiet