Ornith-1.5-9B — NVFP4 GGUF with MTP head (tiyuvta serving artifact)

NVFP4 (4-bit e2m1, per-16 FP8-e4m3 scales) GGUF of ornith-ai/Ornith-1.5-9B, quantized from the official BF16 safetensors release (token embeddings and output head Q5_K, norms F32). 18.8 GB BF16 → 5.4 GB. Built on request (discussion) as the 9B sibling of Ornith-1.5-35B-A3B-NVFP4-MTP-GGUF.

The MTP head in this file is a graft, and the card says so. The official Ornith-1.5-9B checkpoint declares an MTP head in its config (mtp_num_hidden_layers: 1) but ships no mtp.* tensors — 760 tensors where its base has 775. This artifact takes the trained MTP head from Qwen/Qwen3.5-9B (Apache-2.0, the base model this fine-tune derives from — the two checkpoints are tensor-census-identical except for those 15 head tensors) and grafts it onto the unchanged official Ornith trunk at blk.32 (nextn_predict_layers=1). A draft head can never change output — the target verifies every drafted token — it only moves acceptance and speculative speed. Measured acceptance with the grafted head is in the tables below; the speculative stream is gated token-identical to plain greedy decode at every K.

Built as a serving artifact for the tiyuvta serving engine on RTX Blackwell (sm_120a), with per-request exactness gates: speculative, graphed, and batched serving are gated byte-identical to plain decode.

NVFP4 is not an upstream llama.cpp tensor type: this file runs on the serving engine and on the NVFP4 branch of avifenesh/llama.cpp. For upstream-llama.cpp/Ollama use, the official BF16–Q8_0 GGUFs are the right pick (they carry no MTP head).

• Hosted inference: https://inference.tiyuvta.ai
• Context: 262,144 native; chat template embedded — byte-identical to the repo's chat_template.jinja (sha256 9dd2fbd270fe…44e8fd235; XML tool calling + <think> reasoning)
• Vocabulary: 248,320 (text→text serving; the upstream vision tower is not in this artifact)

Provenance and verification

Measured acceptance (exactness harness, not a throughput bank)

Greedy run-spec, serving engine v0.104.1, RTX 5090 Laptop. Acceptance = accepted/drafted over the run; the speculative stream is token-identical to plain decode in every row (that is the gate). This laptop card is thermally limited, so no tok/s numbers are published here — measure on your own hardware.

What ships here, file by file

Where the masked head lives, and what the mask does

mtp-Ornith-1.5-9B-NVFP4-frspec-sxc32768.gguf contains:

"Masked head" means: the draft proposes tokens only from the top-32,768 ids ranked by how often this model itself emits them (FR-Spec-style d2t trim). The ranked distribution is 100% model-generated — 138,065 own-generated tokens over real agentic-session prompts; external text was used as prompts only. topN = 32768 fixes the masked head's shape — a different topN is a different artifact.

The mask can never change output. Verification runs on the target's full vocabulary, so a trim moves draft acceptance (speed) and nothing else — run-spec asserts the masked-head speculative stream token-identical to plain decode, and that gate is PASS on this artifact (file sha256 6b027b05ba92dfda2f9786a1a970b028b8475345223b9dc85b25d1e51224775b). On the build's --validate A/B the trim cost 0.5 acceptance pts against the untrimmed embedded head (0.429 vs 0.434 over the held-out sweep) for a 7.6× smaller draft-head read — verdict GOOD on the measurement card; end-to-end tok/s decides it on yours.

How the files are used (tiyuvta serving engine)

This artifact runs on the tiyuvta serving engine. Three ways to draft, all gated byte-identical to plain decode:

1. Trunk alone. The grafted head is embedded in Ornith-1.5-9B-NVFP4-Q5K-mtp.gguf, so the trunk drafts from its own full 248,320-row head with no extra file.
2. Trunk + pre-trimmed masked head. mtp-Ornith-1.5-9B-NVFP4-frspec-sxc32768.gguf attaches to the trunk as an external draft head (blk.32, head_vocab=32768, d2t map).
3. Ranks file only. ornith15-9b-ranks-sxc-32768.gguf.txt (or the ranks .gguf) self-trims at load: the engine gathers the 32,768 ranked rows from the trunk's own output.weight bytes (byte-level row gather, zero requant).

Note on safetensors trunks: the official Ornith-1.5-9B safetensors checkpoint ships no MTP head, so there is nothing there for a ranks file to trim; speculative drafting for this model comes from this repo's GGUFs.

Other runtimes

The trunk is a standard GGUF layout for the qwen35 architecture with a NextN block; engines that read NextN heads see the grafted MTP out of the box. The masked draft files carry the d2t tensor layout (see ggml-org/llama.cpp#25187); outside the serving engine, draft from the trunk's embedded full head instead of the masked files.

Build your own ranks (and your own masked head)

Rank files are vocab + distribution artifacts of the exact serving model: derive fresh ranks from the model's own generations for every model and every requant. The recipe:

1. Ranks from the model's OWN generations (corpus text is prompts only): count how often the model emits each token id, keep the top 32,768, and write them as a d2t container (.gguf) and as one id per line (.txt).
2. Extract the MTP block byte-verbatim from the trunk, trim the head to those ranks, and requantize (NVFP4 head + Q4_K_M block, the measured-best order) with llama-quantize.

The measured laws: per model, per requant (foreign ranks measured −12 acceptance pts on an identical tokenizer); chat template ON if you serve chat (a raw-derived rank set once cost −15 acceptance pts); corpus floor ≥ 4× topN own-generated tokens (131,072 for a 32,768 head); validate before trusting (frspec-owngen … --validate A/Bs trimmed vs untrimmed end-to-end).

Built with convert_hf_to_gguf.py + llama-quantize (NVFP4 branch) + frspec-owngen + tools/make-trimmed-draft.sh — the same recipe documented above.