Spark-X2.5-4B — ASHQ1 Quantization
ASHQ1 quantization of Spark-X2.5-4B (dense 4.1B, spark2_5 arch).
Status: published. Custom Spark-X2.5-4B imatrix (216 tensors, 36 layers). A second imatrix from the Qwen3.8 calibration set gives a byte-identical tier distribution at 4000 MiB — results are calibration-robust.
Architecture (why ASHQ1 fits as-is)
- Dense decoder, 36 blocks (per imatrix), hidden 4096, 32 heads GQA
- Hybrid attention: 1 full-attention + 3 sliding-window layers repeating — SWA affects KV-cache, not quantization; no special tensors
- Fused
q_k_v_proj(like qwen35) → tied groups work out of the box - MLP
ffn_gate/ffn_up/ffn_down, RMSNorms (attn_normper block) - No MTP, no MoE, no SSM — plain dense path
- Vocab 131072, native 1M context, BF16, Apache-2.0
Imatrix tensor types (attn_gate, attn_qkv, attn_output, ffn_gate, ffn_up, ffn_down) are all covered by the existing class map — no new mappings needed. Preset spark2_5 added, detected via general.architecture metadata.
Quants
| File | Size | PPL (wiki.test.raw, ctx 1024) | Description |
|---|---|---|---|
Spark-X2.5-4B-BF16-Q8_0.gguf | 4.1 GB | 30.86 ± 0.29 | Uniform Q8_0 reference |
Spark-X2.5-4B-BF16-ASHQ1-6000.gguf | 4.0 GB | 30.84 ± 0.29 | Matches Q8 at slightly smaller size |
Spark-X2.5-4B-BF16-ASHQ1-4000.gguf | 3999 MiB | 31.25 ± 0.30 | Recommended: +0.4 PPL for −100 MiB |
Spark-X2.5-4B-BF16-TOPDOWN-4000.gguf | 4005 MiB | 32.12 ± 0.31 | Top-down variant — bottom-up wins |
Spark-X2.5-4B-BF16-ASHQ1-2600.gguf | 2605 MiB | 32.85 ± 0.31 | Tight: Q4/Q5/Q6 mix, no sub-4-bit |
Spark-X2.5-4B-BF16-ASHQ1-2000.gguf | 2002 MiB | 61.33 ± 0.62 | Floor demo — lands between uniform Q3_K_M (44.88 @ 2.1 GB) and Q2_K (95.78 @ 1.4 GB): better than Q2 at near-Q3 size, but the cliff below 2600 is real |
Spark-X2.5-4B-BF16-Q3_K_M.gguf | 2.1 GB | 44.88 ± 0.45 | Uniform low-bit reference (not uploaded) |
Spark-X2.5-4B-BF16-Q2_K.gguf | 1.4 GB | 95.78 ± 0.99 | Uniform low-bit reference (not uploaded) |
PPL measured with stock llama.cpp (llama-perplexity, -ngl 99 -c 1024 -b 512 --seed 7) on wikitext-2-raw wiki.test.raw. Absolute values are inflated (model reads raw text without its jinja template); relative comparison is valid — all files measured identically. Note: spark needs -c 1024 — at -c 512 the same Q8 scores ~2× worse (context fragmentation).
⚠️ PPL is a canary, not a rank column: PPL below the BF16 base flags over-confidence (sharpening), not quality. Cross-checking all PPL-only wins on a KLD-vs-BF16 battery (with Soulfate24) — rankings below are provisional until KLD-scored.
Notes
- Source BF16:
XHToken/Spark-X2.5-4B-GGUF(BF16 conversion) - Imatrix: custom Spark-X2.5-4B imatrix (216 entries, 36 layers)
- Runs on stock llama.cpp (native
spark2_5support since Sep 2026, no fork needed) - Quantization and tuning by wepiqx