Noema-2B — GGUF
GGUF builds of NoemaAI-labs/Noema-2B,
a ~2B reasoning model post-trained from Qwen/Qwen3.5-2B
(hybrid Gated-DeltaNet architecture), for use with llama.cpp and
compatible runtimes.
For the full model description, benchmarks, training recipe, and limitations, see the base model card.
⚠️ Requires Qwen3.5 (qwen35) architecture support in llama.cpp
This model uses the Qwen3.5 hybrid Gated-DeltaNet architecture. It
will only load in a llama.cpp build that includes the qwen35
architecture (linear + full attention layers). On an older
build you will get an "unknown architecture" / unsupported-arch error.
Use a recent llama.cpp with Qwen3.5 support and build from source if
your package manager's version is too old.
Files
| File | Quant | Size | Notes |
|---|---|---|---|
Noema-2B-F16.gguf | F16 | 3.5 GB | Full precision; conversion reference. |
Noema-2B-Q8_0.gguf | Q8_0 | 1.9 GB | Near-lossless; highest-fidelity quant. |
Noema-2B-Q6_K.gguf | Q6_K | 1.5 GB | Very high quality, small quality loss. |
Noema-2B-Q4_K_M.gguf | Q4_K_M | 1.2 GB | Recommended quality/size balance. |
All quants were produced from the same F16 conversion of the release
weights and smoke-tested (load + generation) on each file.
Usage
Noema is tuned and evaluated in non-thinking mode, which is the
default the embedded chat template applies. Pass --jinja so that
template is used, and the model runs in its trained (non-thinking) regime
automatically — no extra flags or prompt tags are needed.
Recommended sampling (the config Noema was evaluated under):
--temp 0.7 --top-p 0.8 --top-k 20 --presence-penalty 1.5.
Avoid --temp 0 / greedy decoding — this model family is documented
to enter repetition loops under greedy sampling.
# One-shot, single-turn (non-thinking):
llama-cli -m Noema-2B-Q4_K_M.gguf --jinja -cnv -st \
-p "A farmer has 17 red apples and 25 green apples. How many in total?" \
-n 256 --temp 0.7 --top-p 0.8 --top-k 20 --presence-penalty 1.5
# Or run an OpenAI-compatible server (set the sampling params client-side,
# or pass the same flags here as server defaults):
llama-server -m Noema-2B-Q4_K_M.gguf --jinja -c 4096 \
--temp 0.7 --top-p 0.8 --top-k 20 --presence-penalty 1.5
License
Apache-2.0, inherited from the Qwen/Qwen3.5-2B base model.