Holo-3.1-35B-A3B — GGUF with restored MTP head, vision projector and native imatrix

Quantizations of Hcompany/Holo-3.1-35B-A3B prepared with three properties not otherwise available in combination:

1. the model's native multi-token-prediction head restored, enabling speculative decoding;
2. the vision projector published alongside, preserving computer-use capability;
3. an importance matrix calibrated on the model's own agent-loop output.

All quantized files derive from a single bf16 master, which is also published.

1. Quantized models

Every file carries the MTP head at blk.40, and all are imatrix-guided.

2. Supporting artifacts

3. Preparation

3.1 MTP head restoration

Holo-3.1-35B-A3B ships without multi-token-prediction tensors. The parent model was established by direct weight comparison rather than metadata, the upstream card being internally inconsistent on the point (prose states "Qwen 3.5 family"; YAML frontmatter states Qwen/Qwen3.6-35B-A3B):

Tensor cardinality corroborates it: Holo-3.1 contains 1026 tensors, Qwen3.6-35B-A3B contains 1045, and the difference is exactly the 19 mtp.* tensors. Holo-3.1-35B-A3B is therefore Qwen3.6-35B-A3B with the MTP head removed and then fine-tuned; the head here is restored from the parent model, not adapted from an unrelated one.

The 19 mtp.* tensors (844.6 M parameters) were grafted at safetensors level before conversion and asserted byte-identical to the donor as a precondition of proceeding:

donor head sha256  faac91f15cbe54475faa2578bedc46a7c29a947b8a3e7ef3ecd376ae079826ab

The head is deliberately not fine-tuned. Prior work on KAT-Coder-V2.5-Dev fine-tuned the equivalent head twice on model-native rollouts; both attempts reduced draft acceptance from 73% to 45–47%.

Measured draft acceptance: 85.0% (577/679 drafts) at 204 tokens/s, with the vision projector loaded, reproduced on two machines. The same head on KAT-Coder-V2.5-Dev reaches 73–76%; the difference is attributable to lineage proximity. Measurement used a mixed code/edit/structured prompt set rather than a GUI-agent benchmark.

3.2 Vision projector

Holo-3.1 contains 333 model.visual.* tensors. convert_hf_to_gguf.py emits the text model only, so the projector was exported separately via --mmproj. A deployment without it is text-only and cannot ground on screenshots.

3.3 Importance matrix

Calibrated on a tripartite corpus, evenly interleaved:

Rollouts were generated by the model itself at temperature 0.7 with no presence penalty, then rendered through Holo-3.1's own chat_template.jinja, whose default strips <think> blocks from assistant turns preceding the final user query — matching served context rather than an idealised transcript.

The corpus is interleaved rather than concatenated, so any prefix preserves the 40/40/20 ratio, and is sized to the calibration budget: 980 chunks × 512 tokens = 501,760 tokens against a corpus of 2,006,278 characters, so the corpus is consumed in full. The Holo share is selected round-robin across task goals, so every distinct goal is represented before any goal repeats.

Matrix verification: 510 tensors, no non-finite values, no all-zero tensors, no zero-activation tensors, 430 of 442 quantizable tensors covered.

3.4 Tensor maps

Each tier's map was read from that tier's own published GGUF header by ranged HTTP request, from Qwen3.6-35B-A3B sources — unsloth/Qwen3.6-35B-A3B-MTP-GGUF and mudler/Qwen3.6-35B-A3B-APEX-MTP-GGUF — Holo-3.1 sharing that architecture exactly. No map was inferred, and none was shared between tiers. Each was validated against the model's real tensor list for double-matched patterns and uncovered quantizable tensors; all nine returned zero of each.

Role-level precision allocation is the design of Unsloth and mudler respectively. The four UD tiers reproduce Unsloth's published file sizes exactly (18.21 / 22.85 / 25.54 / 30.01 GB), confirming faithful replay.

v2D-lite is applied to APEX-I-Compact only: it raises attn_k and attn_v on the ten full-attention layers, a net-zero-byte reallocation with effect only where the parent map leaves attention at Q4_K.

4. Deployment

llama-server \
  -m Hcompany_Holo-3.1-35B-A3B-MTP-UD-Q4_K_XL.gguf \
  --mmproj Hcompany_Holo-3.1-35B-A3B-mmproj-F16.gguf \
  -c 65536 -fa on --jinja \
  --spec-type draft-mtp,ngram-mod \
  --spec-draft-n-max 1 --spec-draft-n-min 0 --spec-draft-p-min 0.75 \
  --spec-ngram-mod-n-min 8 --spec-ngram-mod-n-max 24 --spec-ngram-mod-n-match 48

Speculative decoding and the vision projector operate together; llama.cpp rejects that combination only when draft and target differ in n_embd_inp, and here both are the same trunk.

--spec-draft-p-min 0.75 is the highest-impact setting: restricting drafting to high-confidence positions is what makes the head profitable.

Constraints imposed by --mmproj, per tools/server/server-context.cpp:

• ctx_shift is disabled
• cache_reuse is disabled
• prompt checkpoints are not created following image chunks

5. Building additional tiers

hf download gbuzhf/Holo-3.1-35B-A3B-MTP-GGUF --include "BF16/*" --local-dir .
llama-gguf-split --merge BF16/Hcompany_Holo-3.1-35B-A3B-BF16-MTP-00001-of-00002.gguf master.gguf
llama-quantize --imatrix Hcompany_Holo-3.1-35B-A3B-MTP-imatrix.gguf \
               --tensor-type-file your_map.txt master.gguf out.gguf Q4_K_M

6. Limitations

1. The importance matrix is text-only. --mmproj is registered in llama.cpp's common/arg.cpp for LLAMA_EXAMPLE_{MTMD,SERVER,CLI,TTS} and not for IMATRIX, so no projector output can enter calibration in any current build. The rollouts capture the tool-call schema, ChatML framing and observation loop verbatim; they do not capture visual grounding.
2. blk.40 is not covered by any importance matrix. llama-imatrix does not execute the MTP head during a forward pass, so that block is quantized unguided in all builds.
3. The blend ratio was selected analytically, not empirically. A 50/50 two-way control and a KL comparison against the bf16 reference were planned but not executed.

7. Credits

H Company — Holo-3.1 · Qwen — Qwen3.6-35B-A3B and the MTP head · Unsloth — Dynamic 2.0 tensor maps · mudler — APEX tensor maps · bartowski — calibration corpus · llama.cpp

Licensed Apache-2.0, inherited from the base model.