Qwen3.8-27B Q4_K_M GGUF
This repository contains one independently downloadable Q4_K_M GGUF of
Qwen/Qwen3.8-27B. It is an unmodified vanilla quantization, not a fine-tune,
merge, ablation, alignment change, or chat-template modification.
The source weights are pinned to 1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0. The official checkpoint's
internal Qwen3_5ForConditionalGeneration / qwen3_5 architecture identifier
does not mean this is a Qwen3.5 model.
Files and conversion
- Main model:
Qwen3.8-27B-Q4_K_M.gguf(16.811 GB decimal) - Vision projector:
mmproj-Qwen3.8-27B-F16.gguf - Complete repository size: 17.751 GB decimal
- Quantization:
Q4_K_Musing llama.cpp K/IQ quantization - llama.cpp revision:
5f754ea0e2fd21e1213db7ebebfd65d938d9d69c - Calibration: none for K-quants; representative local prompts where required by IQ conversion
- Source inventory: 1,199 tensors, including 333 vision tensors and 15 source MTP tensors
Validated component status
- Text generation: passed
- Tool calling: passed all five native-format cases
- Vision and video: passed all three deterministic local-image cases
- Thinking controls and chat template: preserved and checked against the locked source
- Hybrid Gated DeltaNet/full-attention structure: retained
- MTP tensors and metadata: retained; speculative acceleration is not advertised
- Per-file runtime smoke: passed at 13.640331732867743 generated tokens/s on the validation host
- Weight SHA-256:
115a859329b5f4e876bed5ed49225d8097d055783a81945004a80af2fb2703f8 - Validation lineage: the same bytes passed in the combined release at
f519a212d6c15cd3292b6ca835dd8ebf235642c0
validation_result.json records the exact per-file smoke result and its inherited
aggregate functional gates. These split repositories do not claim a new benchmark
run; they preserve byte-identical, already-validated artifacts.
Inference
git clone https://github.com/ggml-org/llama.cpp
cd llama.cpp && git checkout 5f754ea0e2fd21e1213db7ebebfd65d938d9d69c
cmake -S . -B build -DGGML_METAL=ON -DCMAKE_BUILD_TYPE=Release
cmake --build build --config Release -j
hf download Chungulus/Qwen3.8-27B-Q4_K_M-GGUF --local-dir ../qwen38-q4_k_m
cd ../qwen38-q4_k_m
../llama.cpp/build/bin/llama-mtmd-cli \
-m ./Qwen3.8-27B-Q4_K_M.gguf \
--mmproj ./mmproj-Qwen3.8-27B-F16.gguf \
-p 'Describe the image.' --image ./image.png
Allow at least approximately 21 GB of available memory for the model, vision projector, and modest runtime overhead; KV-cache needs grow with context. This is planning guidance, not a measured peak-memory guarantee.
Use the source chat-template controls for enable_thinking, reasoning_effort,
and preserve_thinking, together with Qwen's native tool-call format.
Limitations
Quantization can reduce quality, particularly at lower bit widths. The longest recorded validation prompt was 73 tokens; do not infer that the architectural maximum context was tested. A runtime must support the Qwen3.8 hybrid language graph, vision tower/projector, tokenizer, and retained MTP metadata rather than loading only a single language tensor.
License and attribution
The parent model and this vanilla quantization use the source model's Apache-2.0 license. See Qwen/Qwen3.8-27B.