Ornith-1.5-9B GGUF

GGUF quantizations of https://huggingface.co/ornith-ai/Ornith-1.5-9B for use with llama.cpp, LM Studio, Ollama (GGUF import), KoboldCpp, and other GGUF-compatible runtimes.

About the Base Model

Ornith-1.5-9B is a 9B-parameter Qwen3.5-family model using a hybrid architecture consisting of:

• 32 transformer layers
• Hybrid linear attention (Gated DeltaNet) and full attention
• MRoPE positional encoding
• Dense architecture (no routed experts)

These GGUF files are text-only conversions intended for inference in llama.cpp-compatible software.

Conversion Notes

• Converted using upstream llama.cpp with native Qwen3.5 support.
• The model was converted directly to Q8_0 GGUF and all other quants were generated from that Q8_0 source.
• Importance matrix (imatrix) calibration was generated using WikiText-2 and applied to IQ quantizations.
• The original Hugging Face configuration advertises an MTP/next-token prediction layer that is not present in the released weights. GGUF metadata was patched so the layer count matches the actual tensors contained in the model.

Available Quantizations

Standard Quants

• Q2_K
• Q3_K_S
• Q3_K_M
• Q3_K_L
• Q4_0
• Q4_1
• Q4_K_S
• Q4_K_M
• Q5_K_S
• Q5_K_M
• Q6_K
• Q8_0

IQ Quants

• IQ2_M
• IQ3_XXS
• IQ3_XS
• IQ3_M
• IQ4_XS
• IQ4_NL

Quant Recommendations

Usage with llama.cpp

./llama-cli \
  -m Ornith-1.5-9B-Q4_K_M.gguf \
  -p "Explain quantum computing in simple terms."