[!IMPORTANT] Compatibility: These GGUF files require a
llama.cppbuild with K2 Horizon architecture support. Until upstream support lands, use the MBZUAI-IFM fork.
K2-Horizon-MoVA-36B-A4B GGUF
GGUF quantization of IFM/K2-Horizon-MoVA-36B-A4B, a sparse Mixture-of-Experts model with Mixture-of-Values attention (MoVA), 36B total parameters, and approximately 4B active parameters per token. The source checkpoint supports a native context length of 524,288 tokens (512K).
Benchmarks
Benchmark results reported by IFM for the original K2-Horizon-MoVA-36B-A4B checkpoint.
GGUF files
| Quantization | File | Size |
|---|---|---|
| Q3_K_M | K2-Horizon-MoVA-36B-A4B-Q3_K_M.gguf | 17.66 GB |
| Q4_K_M | K2-Horizon-MoVA-36B-A4B-Q4_K_M.gguf | 22.37 GB |
| Q4_0 | K2-Horizon-MoVA-36B-A4B-Q4_0.gguf | 21.28 GB |
| Q5_K_M | K2-Horizon-MoVA-36B-A4B-Q5_K_M.gguf | 26.44 GB |
| Q6_K | K2-Horizon-MoVA-36B-A4B-Q6_K.gguf | 30.77 GB |
| Q8_0 | K2-Horizon-MoVA-36B-A4B-Q8_0.gguf | 39.83 GB |
The model is text-only; no vision projector is required. SHA-256 checksums are provided in SHA256SUMS.txt.
Chat template
Each GGUF embeds the llama.cpp-compatible chat template. chat_template.jinja is a matching external copy for tools that require one. The original source template is retained as chat_template.upstream.jinja for runtimes with full Jinja support.
xml is the default tool-call format. Use --chat-template-kwargs to select json or xml_typed when required.
Usage
Use the IFM K2 Horizon llama.cpp fork. The example below uses a practical 128K context; use -c 524288 when available memory permits.
llama-cli \
-m K2-Horizon-MoVA-36B-A4B-Q4_K_M.gguf \
-c 131072 --jinja \
--temp 1.0 --top-p 0.95
Source
- Source model: IFM/K2-Horizon-MoVA-36B-A4B
- Source revision:
05cab0a - Source license: Apache-2.0