K2-Horizon-3.7B — Pollard
Pollard shrank this model: 10.12 GB (f16) → 2.75 GB — 73% smaller, 3.7× down.
The smallest rung here; larger, higher-fidelity rungs are listed below.
format this model's size f16 10.12 GB Q8_0 ~5.36 GB Q6_K ~4.15 GB Q4_K_M ~2.93 GB PollardMix (this repo's IQ3_S) 2.75 GB
Pollard builds of IFM/K2-Horizon-3.7B made with Pollard Weights — a ladder of measured-allocation quants (bits placed by per-layer sensitivity, not a uniform crush).
These files need the MBZUAI-IFM llama.cpp fork. This model's architecture (k2-horizon) is not one upstream llama.cpp knows, so stock llama.cpp -- and therefore Ollama and LM Studio -- cannot load them whatever the quant types are. The quants themselves are ordinary K-quants.
Model details
| Parameter count | ~5.1B |
| Architecture | k2_horizon |
| Input support | text |
| imatrix | no |
| Perplexity measured | yes — table below |
Which file should I choose?
Every rung is the same weights, sized to a different RAM budget by the measured allocation. Pick the largest one that fits your machine with room for context:
- ~6 GB RAM / VRAM →
Q6_K(4.16 GB). (needs the MBZUAI-IFM llama.cpp fork) near-lossless (+0.08 over f16) - ~5 GB RAM / VRAM →
IQ4_XS(3.33 GB). (needs the MBZUAI-IFM llama.cpp fork) recommended default (+0.14) - ~5 GB RAM / VRAM →
IQ3_S(2.75 GB). (needs the MBZUAI-IFM llama.cpp fork) smallest (+1.24)
Available files (wikitext-2 test, ctx 512)
f16 reference PPL 11.2798...
| file | PPL | size | Mean KLD | notes |
|---|---|---|---|---|
K2-Horizon-3.7B-Pollard-IQ3_S.gguf | 12.5164 | 2.75 GB | — | smallest (+1.24) |
K2-Horizon-3.7B-Pollard-IQ4_XS.gguf | 11.4211 | 3.33 GB | — | recommended default (+0.14) |
K2-Horizon-3.7B-Pollard-Q6_K.gguf | 11.3603 | 4.16 GB | — | near-lossless (+0.08 over f16) |
Measured notes
f16 reference PPL 11.2798.
Measured notes
f16 reference PPL 11.2798..
Download a specific file
pip install -U "huggingface_hub[cli]"
hf download PollardWeights/K2-Horizon-3.7B-Pollard \
--include "K2-Horizon-3.7B-Pollard-IQ4_XS.gguf" --local-dir ./
How to run
This model's architecture (k2-horizon) needs the MBZUAI-IFM llama.cpp fork, so every file here runs there:
llama-cli -m K2-Horizon-3.7B-Pollard-IQ4_XS.gguf -ngl 99 -p "Explain why the sky is blue."
llama-server -m K2-Horizon-3.7B-Pollard-IQ4_XS.gguf -ngl 99
No rung in this repo loads in stock llama.cpp, so Ollama and LM Studio cannot run these files.
ARM / AVX
llama.cpp repacks weights into an interleaved layout at load time for faster inference on ARM and AVX machines — no special file needed, online repacking covers these quants. The old Q4_0_4_4/4_8/8_8 variants are not required.
Errata
general.architectureisk2-horizon, which upstream llama.cpp does not implement, so these files load only in the MBZUAI-IFM llama.cpp fork — the quant types are ordinary and irrelevant to that. Checked withpollard-ggufcheck, which reads the architecture and the tensor types out of the header.- Measured allocation places bits by per-layer sensitivity under a size budget.
- Single machine; replication invited.
Credits & license
- Base model:
IFM/K2-Horizon-3.7B - Quantization tooling: llama.cpp (ggml-org)
- Method + tooling: Pollard Weights — measure first, no claim before a number.
- License:
apache-2.0, inherited from the base model.
Built with Pollard Weights — frontier models, small hardware, no compromise.