Ornith-1.5-9B — AGENT — ROCmFP4 / ROCmFPX GGUF
The 8-bit AGENT tier of ornith-ai/Ornith-1.5-9B — ftype 115 Q8_0_ROCMFPX_AGENT, keeping more attention-output tensors at higher precision for tool-calling.
Built for AMD Strix Halo (gfx1151) — Ryzen AI MAX+ 395, 128 GB unified memory, ROCm 7.13.0 — using the ROCmFPX llama.cpp fork, which adds AMD-native FP4/FP8 tensor types that mainline llama.cpp does not have.
⚠️ These files require a ROCmFPX-capable llama.cpp build. They will not load in stock llama.cpp / Ollama / LM Studio — the
Q4_0_ROCMFP4_*andQ*_0_ROCMFPX*tensor types are not in mainline.
Variants in this repo
| file | ftype | size | BPW | token_embd | output.weight | decode |
|---|---|---|---|---|---|---|
Ornith-1.5-9B-Q6_0_ROCMFPX_AGENT.gguf | 114 | 7.90 GiB | 7.58 | Q8_0 | Q8_0 | 22.13 t/s |
Ornith-1.5-9B-Q8_0_ROCMFPX.gguf | 111 | 8.67 GiB | 8.31 | Q8_0 | Q8_0 | 22.00 t/s |
Ornith-1.5-9B-Q8_0_ROCMFPX_AGENT.gguf | 115 | 8.77 GiB | 8.41 | Q8_0 | Q8_0 | 22.16 t/s |
Which to pick: the AGENT tiers when tool-calling fidelity matters more than speed. ⚠️ All three 8-bit tiers run at ~22 t/s — about 60% the speed of the 4-bits — and Q6_0_ROCMFPX_AGENT is not a middle rung at 7.58 BPW. The AGENT routing's benefit shows up with a speculative drafter, and this model has none, so it cannot demonstrate that advantage here.
Head protection — verified in the file, not assumed
tie_word_embeddings is false on this model, so output.weight is a real standalone tensor and --output-tensor-type genuinely bites. Every artifact was re-opened after quantization and its header read back with an independent parser:
| ftype | token_embd.weight | output.weight |
|---|---|---|
| 106 STRIX_LEAN | Q5_K | Q6_K |
| 104 FAST_COHERENT | Q6_K | Q6_K |
| 102 COHERENT | Q6_K | Q6_K |
| 114 / 111 / 115 | Q8_0 | Q8_0 |
⚠️ Note the LEAN row: it keeps the Q5_K token embeddings that make it lean while still protecting the output head at Q6_K. ftype 106 does not do this on its own — left to its defaults it emits a 4-bit output.weight. Head protection has to be requested and then verified in the file; an ftype name is not a guarantee.
Measured — not estimated
Hardware: AMD Ryzen AI MAX+ 395 (Strix Halo, gfx1151), 128 GB unified, ROCm 7.13.0. Idle box, 2 warm-ups discarded, median of 5, 300 tokens, identical prompt across every sample.
| quant | run 1 | run 2 | run 3 | run 4 | run 5 | median |
|---|---|---|---|---|---|---|
| 114 | 22.40 | 22.22 | 22.13 | 22.07 | 22.03 | 22.13 |
| 111 | 21.94 | 21.98 | 22.01 | 22.00 | 22.02 | 22.00 |
| 115 | 22.05 | 22.16 | 22.13 | 22.17 | 22.18 | 22.16 |
Speculative decoding
⛔ Not available for this model. config.json declares mtp_num_hidden_layers: 1, but the published safetensors contain no MTP weights and the converted GGUF has no nextn_predict_layers key — we checked both. No draft head ships with this model, and no GGUF drafter is published for it. The decode figures below are therefore the deployed numbers, with no speculative multiplier left on the table.
Source — byte-verified, not re-converted
Quantized from ornith-ai/Ornith-1.5-9B-GGUF → Ornith-1.5-9B-BF16.gguf, 17,920,696,768 bytes. The vision projector mmproj-Ornith-1.5-9B-BF16.gguf (921,704,416 bytes) is included. Both byte-verified against the Hub before quantization. No re-conversion from safetensors.
Architecture: Qwen3_5ForConditionalGeneration / qwen35, 427 tensors, 32 layers, hidden 4096, vocab 248,320 — a hybrid stack (24 gated-deltanet + 8 full-attention layers), not a pure dense transformer and not an MoE.
Verification
Every artifact: loaded at -ngl 999 -c 4096 -fit off -fa on, 3/3 correctness (17×23 → 391, capital of Japan → Tokyo, days in 2024 → 366) asserted on both content and reasoning_content, and 4/4 vision on a four-quadrant colour image via the mmproj.
File sizes were checked against --dry-run projections; the header delta is constant across artifacts, which is the signature of complete, untruncated files.