K2-Horizon-32B MLX 6-bit
Uniform 6-bit quantization of IFM/K2-Horizon-32B, a 32B dense decoder-only model with 512K context. Converts to ~24 GB, fitting on 64 GB Apple Silicon with room to spare.
Upstream model: IFM/K2-Horizon-32B by Institute of Foundation Models, released under Apache 2.0.
Conversion: Quantized to MLX format using Hermes Agent with mlx-lm.
Architecture
K2-Horizon-32B is the dense member of the K2-Horizon family — not MoE, not MoVA. A straightforward 32B decoder with 64 layers, hidden dim 5120, and 64 attention heads (8 KV heads).
| Spec | Value |
|---|---|
| Total params | 32B (~35B with embeddings) |
| Architecture | Dense |
| Layers | 64 |
| Hidden dim | 5120 |
| Attention heads | 64 (8 KV) |
| Context length | 512K native |
| Reasoning effort | Always high |
| BF16 size | ~64 GB |
| Quantized size | ~24 GB |
Note: This is the Stage 1 checkpoint. A final checkpoint with improved results is expected.
Quickstart
pip install -U mlx-lm
python3 -m mlx_lm.generate \
--model hermitdave/K2-Horizon-32B-MLX-6bit \
--prompt "Explain why long-context evaluation is difficult." \
--max-tokens 512 --temp 1.0 --top-p 0.95
Reasoning
K2-Horizon-32B is a reasoning model. Always use reasoning_effort="high":
from openai import OpenAI
client = OpenAI(base_url="http://localhost:8000/v1", api_key="EMPTY")
response = client.chat.completions.create(
model="hermitdave/K2-Horizon-32B-MLX-6bit",
messages=[{"role": "user", "content": "Explain step by step."}],
extra_body={"chat_template_kwargs": {"reasoning_effort": "high"}},
)
print("Reasoning:", getattr(response.choices[0].message, "reasoning_content", None))
print("Answer:", response.choices[0].message.content)
oMLX Patch
K2-Horizon requires oMLX v0.6.4+ with the K2-Horizon support patch:
- Registers
k2_horizonmodel type - Recognizes IFM thinking tags (
<ifm|think>,<ifm|think_fast>,<ifm|think_faster>) - Parses IFM tool call format (
<ifm|tool_call>,<ifm|arg_key>,<ifm|arg_value>) - Strips tags in streaming and non-streaming paths
Benchmark Results
| Benchmark | Score |
|---|---|
| tau3 Banking Agentic | 22.5 |
| Terminal-Bench 2.1 | 36.6 |
| SciCode | 30.2 |
| Humanity's Last Exam | 22.8 |
| GPQA Diamond | 82.3 |
| AA-LCR (long-context) | 65.3 |
See the model card for full results.
Citation
@misc{k2horizon2026,
title = {Introducing K2 Horizon: Frontier Performance, Radically Open},
author = {{IFM Team}},
year = {2026},
url = {https://ifm.ai/blog/k2/},
}
License
Apache 2.0 (same as upstream).