Qwen3.8-27B-STQ

Model family

This repository publishes controlled GGUF quantizations of Qwen/Qwen3.8-27B. Every tier was produced from the same BF16 GGUF with the same pinned llama.cpp build and importance matrix. This is a measurement study, not a new codec claim.

Point llama.cpp at the first shard; the remaining shards are discovered automatically.

Measured quality

BF16 baseline: 696/800 = 87.00%. Two independent paid runs reproduced the same file hash and item-level correctness vector. Results use a preregistered 800-item stratified MMLU+ARC letter-logprob harness, seed 22, 10,000 paired bootstrap samples, exact two-sided McNemar, and a +2.00 pp non-inferiority margin.

Perplexity is monotone with bit width even though Top-1 is not. The IQ4/IQ3 ordering is therefore treated as a property of this 800-item sample, not a broad quality claim.

BF16 token agreement · unmatched protocols

Pacific agreement is measured from the completed 800-item runs: for each fixed MMLU+ARC prompt, the candidate's raw pre-sampling Top-1 token ID is compared with the BF16 Top-1 token ID. All five evidence files share the same ordered item fingerprint and pinned llama.cpp commit. This is one next-token position per prompt; it is not task accuracy or generation quality.

The Unsloth provider chart also reports BF16 next-token Top-1 agreement, but on a different provider KL-divergence corpus and protocol. Its values above were digitized from the published PNG and rounded to 0.1 pp because no numeric result table is published. The green bars are estimates, and the cross-provider differences are not valid quality deltas. A valid provider comparison still requires running both GGUFs against the same 800 item IDs and scoring code.

The figure is generated with Microsoft Flint Chart 0.5.1 using its Swiss theme. Reproducibility assets: measured result · calculation script · Flint input · compiled Vega-Lite spec · render script.

LiveCodeBench v6 · preliminary

Execution grading is pending. The value below is a hard upper bound, not a measured pass@1 score. No extraction failure was manually repaired.

The best-case cross-protocol gap to the upstream BF16 reference is ≤−3.29 pp. The actual gap can only be larger. A matched BF16 run is required before claiming quantization retention.

The initial 15-item pilot produced two reasoning-only truncations under an 8,192-token total limit. A 16,384-token limit with unrestricted reasoning reproduced the failure. v36 separated an 8,192-token reasoning budget from final-answer capacity; its pilot passed 15/15. Thirteen normally terminated pilot responses were retained, the two truncated responses were regenerated, and 1,040 remaining prompts ran under v36.

Machine-readable audit →

Throughput

Measured on one A100 80GB with full GPU offload, prompt 512 / generation 128, five runs.

Pinned build

The reproducible BF16 text GGUF SHA-256 is 03ab7ad49486af2f111ed8d7616a0f485f9c5032bd6bc2419b84bb3b90f3930f. The BF16 text model and full run evidence are hosted in the artifact dataset.

Usage

llama-cli \
  -m Q4_K_M/qwen38-27b-q4_k_m-00001-of-00003.gguf \
  -p "Explain why low-bit perplexity and task accuracy can disagree."

For vision input, add:

--mmproj mmproj/qwen38-27b-mmproj-bf16.gguf

Merge split shards with llama-gguf-split --merge if a single file is required.

Scope

• MMLU+ARC Top-1, WikiText-2 PPL, and A100 throughput are measured results.
• BF16 next-token agreement is measured at 800 fixed prompt positions from the preserved pre-sampling token IDs; it is not a generation-quality score.
• LiveCodeBench 87.01% is an ungraded ceiling, not pass@1.
• The older v0.26 MLP-only IQ3 result is not pooled with this full-model GGUF study.
• This repository is a research measurement, not a universal deployment recommendation.

License

Apache-2.0, inherited from Qwen/Qwen3.8-27B.