K2-Horizon-0.9B GGUF
Community GGUF quantizations of IFM/K2-Horizon-0.9B.
About K2-Horizon-0.9B
K2-Horizon-0.9B is IFM's compact 0.9B-class dense reasoning model. It has a 128K-token context window with YaRN RoPE scaling and is intended for English and Chinese conversational, reasoning, coding, science, and tool-use workloads. See the official model card for the supported serving stack, prompt conventions, and full evaluation protocol.
Upstream K2-Horizon-0.9B benchmark results; image and results are from the official model card.
This is a quantization-only release. No training, fine-tuning, merging, or weight modification other than GGUF conversion and quantization was performed.
Files
| Quantization | File size (GiB) | A10M generation token/s | Validation | Recommendation / Notes |
|---|---|---|---|---|
| Q8_0 | 1.07 | 281.05 | Load/generate pass | Highest-fidelity option in this release. |
| Q6_K | 0.83 | 310.87 | Load/generate pass | High-fidelity local inference. |
| Q5_K_M | 0.72 | 316.41 | Load/generate pass | Balanced quality and memory. |
| Q4_K_M | 0.62 | 352.89 | Load/generate pass | Recommended starting point for general use. |
| Q3_K_M | 0.51 | 261.54 | Load/generate pass | Lower-memory profile; validate your workload. |
| Q2_K | 0.41 | 326.72 | Load/generate pass | Aggressive low-memory option. |
| IQ2_XS | 0.34 | 328.10 | Load/generate pass | Experimental low-memory profile. |
| IQ1_M | 0.29 | 343.17 | Load/generate pass | Experimental; instruction following and reasoning may degrade. |
| Q1_0 | 0.21 | 446.95 | Load/generate pass | Experimental / legacy minimum-memory option. |
Every file above was loaded and generated non-empty text in the K2 pre-release
llama.cpp runtime on an NVIDIA A10M. The A10M measurements were taken with
single-stream llama-bench, prompt 256, generation 64, batch/ubatch 512,
64 CPU threads, full GPU offload, and three repetitions on an otherwise idle
GPU. Low-bit files—especially Q2, IQ2, IQ1, and Q1—can materially reduce
reasoning, instruction following, and tool-call reliability; test the selected
file with the prompts that matter to you.
K2-Horizon support is pre-release in llama.cpp. The upstream full tool-aware Jinja template uses constructs not yet parsed by the runtime revision used for this release; the reproducibility folder includes the compatible single-turn chat wrapper used for smoke testing. Tool calling is therefore not certified by this GGUF release.
License and attribution
The upstream model is licensed under Apache License 2.0. Preserve upstream attribution and the license when redistributing these derivative artifacts. These are community GGUF quantizations, not an official IFM release or endorsement.
The compact benchmark summary and reproduction script are in
reproducibility/a10m-benchmark.tsv and
reproducibility/benchmark_gpu1.sh.
Checksums for published artifacts and reproducibility inputs are in
SHA256SUMS.txt.