Qwen3.6-35B-A3B Q4_K_M GGUF dequantized BF16 for vLLM
Private Exolabs checkpoint converted from
bartowski/Qwen_Qwen3.6-35B-A3B-GGUF file
Qwen_Qwen3.6-35B-A3B-Q4_K_M.gguf.
The upstream Qwen/Qwen3.6-35B-A3B repository was used for config,
tokenizer, chat template, and generation config only. Weight tensors come from
the Q4_K_M GGUF and were dequantized in fp32, transformed back to HF/vLLM
layout where llama.cpp stores Qwen3.5/3.6 Gated DeltaNet tensors differently,
then cast once to BF16.
Validated on vLLM 0.23.0 with:
VLLM_USE_FLASHINFER_SAMPLER=0 vllm serve . \
--served-model-name qwen36-gguf-q4km-bf16-gdnfix \
--dtype bfloat16 \
--max-model-len 4096 \
--moe-backend triton \
--attention-backend triton_attn \
--gdn-prefill-backend triton \
--trust-remote-code \
--language-model-only
Smoke test output for The capital of France is begins with Paris.