📚 Collection

▶ Archsloth MODELs — every build in the ARCHsloth Q4 grade Qwen3-4B · Qwen3.5-9B · Qwen3.8-27B

Qwen3-4B · ARCHsloth Q4 / Q6 / Q8

Same architecture. Same tensor layout. Same file size, to the byte. Only the rounding differs.

🦥 Try it live, no install → — answering on a CPU-only box, no GPU.

Which file do I want?

Those are measured process allocations (llama-server -ngl 99, single user), not file size plus arithmetic. At 128K context add roughly 18 GB to any row — the cache decides whether a long-context session fits, not the weights.

There is no Q5 file here. Why not.

[measured] Q4_K_M — ten axes, 2,497,280,800 bytes on both sides

KL divergence from the bf16 original. Lower is better. llama-perplexity --kl-divergence, ctx 512, 60 chunks for Korean and English, 40 for the rest. Every ARCHsloth number below comes from one single build.

Ten axes, seven writing systems, ahead on every one — against the same-size file and against the larger one. Lowest separation is 3.8 σ (Hindi); highest is 20.4 σ (Korean).

Three of these axes are in our calibration set: Korean, English, code. The other seven are not. They are the control, so that a build which buys one language with another has somewhere to show it.

[measured] The whole curve, not one point

Nine of those points are real files: six downloaded from unsloth/Qwen3-4B-GGUF, three built here. Nothing is interpolated. Their curve includes the UD-*-XL files, which are larger than the same-named standard file — and our curve is still below theirs. Read it vertically for "same size, closer to the original"; read it horizontally for "same quality, smaller file".

[measured] Q6_K and Q8_0

Same-top-p and RMS Δp move the same direction on every row. Full statistics: EVAL.md. Raw per-run logs: eval/logs/.

Quickstart

# any recent llama.cpp — brew / winget / apt, or LM Studio / Ollama / PocketPal
llama-cli -m Qwen3-4B-AS-Q4_K_M.gguf -p "안녕하세요" -ngl 0 -t 8

# OpenAI-compatible server
llama-server -m Qwen3-4B-AS-Q4_K_M.gguf --host 127.0.0.1 --port 8080 --jinja

No fork. No custom format. The layout is llama.cpp's own, which is the point.

What is actually different

--scheme GGUF:Q4_K_M     optimize the rounding for the quantizer that is used at export
--enable_alg_ext         sign-gradient rounding search (SignRoundV2, arXiv:2512.04746)
--dataset cal_archsloth.jsonl
--iters 200  --nsamples 128  --seqlen 2048

Nothing else. Bit widths, tensor types and the per-layer layout are llama.cpp's standard Q4_K_M / Q6_K / Q8_0 — 216 × Q4_K, 37 × Q6_K, 145 × F32, zero differences from the same-named stock file. Nothing below 4 bits anywhere.

That constraint is the point. A quant that wins because it spent more bits is not a better quant, it is a bigger one.

The calibration set — it ships in this repo

cal_archsloth.jsonl, 555 samples of exactly 9,000 characters each: Korean 44.9 %, English 35.3 %, source code 19.8 %, interleaved at the sample level.

Interleaving is what mattered, not the ratio: moving to strict alternation with the same character mix improved English by 11.4 %. Adding code moved the code axis by 52.9 % and cost 3.4 % of Korean and 5.3 % of English, which is why it is in. Details: CALIBRATION.md.

What we did not win

• Q5_K_M is not in this repository. Our toolchain emits Q5_K with a 5–6 % deviation between the optimized weights and the packed tensor — every tensor, Q5_K only, on every model we tried. The resulting file scores worse than our own Q4_K_M. A file that gets worse when you give it more bits is a defect, not a result. It ships when that is fixed.
• [measured] Speed is a tie. pp512 12,167 vs 12,227 tok/s; tg128 306.0 vs 305.2 tok/s. One H100, three repetitions, alternating order. Same size and same layout, so a tie is the expected answer — we publish it because a number you omit is a number someone else produces.
• Format compliance is not a ranking here. 13 constrained-output items (7-item numbered lists, nested JSON, tool calls): bf16 13/13, ARCHsloth 12/13, the same-size comparison file 11/13, stock 13/13. Zero language collapse, zero runaway repetition, zero empty outputs on any build. Thirteen items cannot rank four models; it can show that none of them broke.
• [pending] Long-context retrieval and agentic tool-use across bit widths. Not measured yet on this model. We publish only what we ran ourselves.

Knobs that did not work, published so nobody spends the compute twice

Reproduce this

# 1. the comparison file, straight from its own repository
huggingface-cli download unsloth/Qwen3-4B-GGUF Qwen3-4B-Q4_K_M.gguf --local-dir rival

# 2. the reference the distance is measured from
python llama.cpp/convert_hf_to_gguf.py Qwen/Qwen3-4B --outtype bf16 --outfile ref-BF16.gguf

# 3. teacher logits, once per evaluation text
llama-perplexity -m ref-BF16.gguf -f eval/eval_ko.txt \
  --kl-divergence-base base_ko.dat -ngl 99 -c 512 --chunks 60

# 4. every candidate against the same teacher, same text, same chunks
llama-perplexity -m <candidate>.gguf -f eval/eval_ko.txt \
  --kl-divergence --kl-divergence-base base_ko.dat -ngl 99 -c 512 --chunks 60

eval/ holds the ten evaluation texts and the per-run statistics. If a number here does not replicate on your hardware, open a discussion and we will correct it.

Lineage

License

Apache-2.0, inherited from the base model. Comparison files are named so you can fetch the exact bytes we measured.