Qwen3-4B-Instruct-2507 Hi-Fi GGUF — fraQtl
Calibration-aware GGUF builds of Qwen/Qwen3-4B-Instruct-2507: byte-matched
to the leading community Q4_K_M with measurably lower divergence from the
original weights, retrieval-verified through the full 262,144-token native
context — plus a Q4_0 edition in the NPU-native fixed format.
Model Conversion Contributor: fraQtl — built by fraQtl
Model Stats:
- Quantization Type: calibration-aware per-tensor Q4_K_M-class (Hi-Fi) and Q4_0 (fixed format) — weights only, GGUF
- Calibration: fraQtl code+math imatrix (included in this repo)
- Maximum context length: 262,144 (native), retrieval-verified below
- Runtime: llama.cpp (CPU/GPU). GGUF works with GenieX unmodified — per the AI Hub team. (NPU/Hexagon execution not claimed — not yet verified.)
Model Download
| Artifact | Size | Class | Target Runtime | File |
|---|---|---|---|---|
| Hi-Fi (max fidelity) | 2.50 GB | Q4_K_M-class | llama.cpp CPU/GPU · GenieX | Qwen3-4B-Instruct-2507-fraQtl-HiFi-Q4_K_M.gguf |
| Hi-Fi Q4_0 | 2.38 GB | Q4_0 (NPU-native format) | llama.cpp CPU/GPU · GenieX | Qwen3-4B-Instruct-2507-fraQtl-HiFi-Q4_0.gguf |
Fidelity receipts (KLD vs the ORIGINAL model, lower is better)
Symmetric top-20 KLD vs a Q8 teacher quantized from the pinned original bf16; identical binary, template, and slices for every arm; 3 runs, drift 0:
| Arm | Bytes | KLD code/math ↓ | KLD general ↓ |
|---|---|---|---|
| fraQtl Hi-Fi Q4_K_M (this repo) | 2,496,879,712 (−0.016%) | 0.025317 (−55.9%) | 0.033753 (−46.4%) |
| MaziyarPanahi Q4_K_M (canonical, credited) | 2,497,280,448 | 0.057372 | 0.063008 |
| unsloth Q4_K_M (credited) | 2,497,281,120 | 0.030018 | 0.042626 |
Long-context retrieval: needle exact-match, 3 depths × 3 keys per context — 18/18 at 8K+32K (×3 runs) and 9/9 at the full 262,144-token native context (×2 runs); per-cell receipts in the canonical repo.
Q4_0 — calibration lever isolated (fixed format)
| Arm | KLD code/math ↓ | KLD general ↓ |
|---|---|---|
| fraQtl Q4_0 (our imatrix) | 0.059665 | 0.080458 |
| naive Q4_0 (no imatrix, same source) | 0.070135 | 0.088598 |
| GenieX-designated Q4_0 (unsloth's build, credited) | 0.060996 | 0.077572 |
Calibration alone: −14.9% code/math, −9.2% general at identical format and source. Vs the GenieX-designated universal asset (a well-made build): we are −2.2% on code/math and +3.7% behind on general — both directions stated. K-quant-on-Hexagon NPU acceleration is unverified by us; the Q4_0 artifact is the NPU-native-format option.
Canonical repo + full receipts
Complete receipts (7 JSONs) and a runnable receipts pack (single-file runner on stock llama.cpp + teacher logit dumps — re-run every number above against any GGUF of this model): fraQtl/Qwen3-4B-Instruct-2507-Hi-Fi-GGUF
License
Apache-2.0 (inherited from the base model).