Qwen 3.8 27B Fast 60L: Voodoo Dynamic Quantization (Voodoo60L-P1G)

Depth-pruned hybrid DeltaNet with handcrafted per-tensor importance quantization and lossless vocabulary pruning.

Revision note (2026-09-15). This card previously reported 100% coherence, a pp512/tg128 table that mixed two different measurement tools, and a context-scaling table carried over from an earlier model. Every number below has been re-measured on the actual artifacts with llama-bench, and every code result is graded by executing the generated function against test vectors. Corrected figures are lower than the originals. See §7.

What this model is

A 60-layer depth-pruned, vocabulary-pruned, per-tensor quantized build of Qwen 3.8 27B that runs fully GPU-resident on a 12 GB card at ~51 t/s, in 5.59 GiB instead of the 6.77 GiB base.

It is a fast conversational and retrieval model. It is not a coding model. Depth pruning costs multi-step code synthesis, and no quantization layout recovers it — see §4 for the executed-code results.

Sizes are stated in both GiB and GB deliberately: llama.cpp prints GiB, file managers print GB, and mixing them silently inflates apparent savings by ~7%.

1. Architecture

arch = qwen35, full_attention_interval = 4. The stack repeats a strict 4-layer super-block, verified from tensor names:

Layer 4k+0..2 : Gated DeltaNet SSM  (14 tensors)
                ssm_a, ssm_alpha, ssm_beta, ssm_conv1d, ssm_dt.bias,
                ssm_norm, ssm_out + attn_qkv + attn_gate + norms + ffn_*
Layer 4k+3    : Full multi-head attention (11 tensors)
                attn_q/k/v/output + attn_q_norm/k_norm + norms + ffn_*

These are gated DeltaNet layers, not Mamba-2. Note attn_qkv belongs to the SSM layers (45 of them at 60L), while attn_q/k/v belong to the 15 attention layers.

Depth pruning must be in multiples of 4 or the super-block periodicity breaks. At 60 layers: 45 SSM + 15 attention = 15 complete super-blocks.

Attention shape, which matters for KV sizing: head_count = 24, head_count_kv = 4, and key_length = value_length = 256 — not 128. A head-dim assumption of 128 understates the KV cache by exactly 2x.

2. Quantization layout

Default llama-quantize silently promotes two tensors when targeting low-bit types, which is what makes naive IQ1_S builds barely smaller than IQ2_XXS:

Those two promotions add +783 MB, which cancelled the −948 MB that an IQ1_S FFN pass actually gave up — a naive IQ1_S 48L build came out only 3.2% smaller than its IQ2_XXS counterpart. Pinning both recovers the budget, which is then spent where it measurably matters:

ssm_out=iq3_xxs      # DeltaNet state projection - protected
ssm_alpha=q8_0       # recurrent step-size gates - protected
ssm_beta=q8_0
attn_qkv=iq1_s       # attention projections - starved
attn_gate=iq1_s
attn_output=iq1_s
attn_k=iq1_s
attn_q=iq1_s
attn_v=iq1_s
ffn_gate=iq1_s       # gate projection - starved
ffn_up=iq2_xxs       # reasoning capacity - protected (iq1_m in the IQ1M build)
ffn_down=iq2_xxs

Verified as applied, per tensor: ssm_out IQ3_XXS x45, ssm_alpha/ssm_beta Q8_0 x45 each, attn_* IQ1_S, ffn_gate IQ1_S x60, ffn_up/ffn_down IQ2_XXS x60 (or IQ1_M), output.weight + token_embd.weight Q3_K at 286.0 MB each.

Build command

llama-quantize --allow-requantize \
  --imatrix imatrix_unsloth.gguf \
  --tensor-type-file tensor_types_voodoo.txt \
  --output-tensor-type q3_K --token-embedding-type q3_K \
  --prune-layers 28,29,30,31 \
  Qwen3.8-27B-UD-IQ2_XXS.gguf Qwen3.8-27B-UD-Voodoo60L.gguf IQ2_XXS
python prune_vocab.py --policy P1G Qwen3.8-27B-UD-Voodoo60L.gguf

llama.cpp remaps imatrix keys through the prune automatically (remap_imatrix inverts the layer mapping and aborts on mismatch), so no manual imatrix surgery is needed. The base IQ2_XXS file has blocks 0..63 and already carries nextn_predict_layers = 0; the NextN draft head ships as a separate file (MTP/mtp-Qwen3.8-27B-Q4_0.gguf). Pruning a layer 64 is a no-op and no header patch is required.

Vocabulary pruning

P1G removes 118,300 unused multilingual tokens (248,320 → 130,020, −47.6%) in quantized space with BPE substring closure, retaining all 256 byte-fallback tokens plus Latin, Greek, math and box-drawing glyphs. This step is genuinely lossless: the 64-layer vocab-pruned model produces byte-identical greedy output to the unpruned baseline.

3. Throughput

llama-bench, -ngl 99 -fa 1 -ctk q4_0 -ctv q4_0 -p 512 -n 128 -r 3, on RTX 4070 12 GB / Ryzen 7 7800X3D / 64 GB DDR5-6000, build b10665-ca3d5a3e1.

All figures come from llama-bench. The [ Prompt: … | Generation: … ] footer printed by llama-cli is not comparable — it times a ~20-token prompt dominated by startup overhead and reports 4–6x lower prefill. Run-to-run variance on this card is about ±3%.

4. Coherence — graded by execution

Every coding task is graded by running the generated function against test vectors in a subprocess (scripts/eval_coherence.py). A model passes only if its code executes and returns correct answers. Greedy decoding (-st --temp 0 --reasoning off); coding tasks get a second, softer prompt if the first yields zero tokens.

Representative failures on the flagship builds:

• Voodoo60L-P1G, coin_change → emits code that raises IndexError: list assignment index out of range; on a terser prompt it instead loops the phrase "Wait, I think I'm overcomplicating this" five times and never emits code.
• Voodoo60L-IQ1M-P1G, coin_change → UnboundLocalError: cannot access local variable 'min_coines' (a corrupted identifier).
• Voodoo60L-IQ1M-P1G, is_palindrome → zero tokens, immediate EOS.

The unpruned 64-layer baseline passes all four, so these are not artifacts of the prompts or the grader — they are the cost of depth pruning.

Where the floor actually is

Depth degradation is gradual from the first block removed, not a clean cliff. Arithmetic and short factual recall survive to 60L; multi-step code synthesis does not survive any depth cut. 52L additionally loses reliable arithmetic; 48L confabulates ("The phrase 10 + 15 is not a standard arithmetic expression").

Voodoo protection does not rescue 48L or 52L. The SSM:attention ratio is fixed at 3:1 by full_attention_interval = 4 at every multiple of 4 — 48L is 36:12, 60L is 45:15, both exactly 1:3 — so the ratio cannot be the variable. What is lost is absolute recurrent capacity, and no precision layout restores width that is no longer instantiated.

5. Context scaling

Measured from llama.cpp's own allocation log (scripts/measure_context.py), Voodoo60L-P1G, -ngl 99 -fa 1, KV q4_0.

KV cache cost: 15 attention layers x 4 KV heads x 256 dim x 2 (K+V) = 30,720 elements/token. At q4_0 (0.5625 B/element including scales) that is 17,280 bytes/token = 16.875 KiB/token — confirmed exactly by the measured 4320.0 MiB at 256k.

token_embd.weight (272.8 MiB) is held host-side at every context by design — it is a gather, not a GPU matmul — so it is not a context-driven spill.

Generation slows sharply with depth, which a single shallow tg128 number hides:

Generation is ~2.4x slower at 128k and ~5.5x slower at 256k than the headline tg128. 256k technically fits, but with no reserve and at single-digit throughput — prefilling that window alone takes tens of minutes on this card. Treat 128k as the practical working ceiling on 12 GB; 256k is a benchmark configuration, not a usable one.

1M context is not reachable on 12 GB: 1,048,576 x 17,280 B = 18.1 GB of KV cache alone at q4_0, against 5.3 GiB of weights on a 12 GiB card. Even 2-bit kvarn2 needs ~7.5 GiB of cache plus weights and does not fit.

6. How to run

llama-cli -m Qwen3.8-27B-UD-Voodoo60L-P1G.gguf \
  -ngl 99 -fa 1 -c 32768 -ctk q4_0 -ctv q4_0 \
  --temp 0.7 --top-p 0.80 --top-k 20 --presence-penalty 1.5

Raise -c to 131072 for long-context work. Above that, verify headroom on your own card first — the 256k configuration leaves no reserve.

7. Known limitations

1. Not a coding model. 2/4 on executed-code evaluation vs 4/4 for the unpruned baseline. Both coding tasks fail on both flagship builds. Use the unpruned model for code synthesis.
2. Repetition loops persist under terse instructions. Protecting ssm_out and the step gates removed the catastrophic unigram attractor seen in naive 32L builds, but phrase-level looping still appears on constrained coding prompts.
3. IQ1M is instruction-fragile. It returns zero tokens on some terse prompts ("Code only.") while answering the same request phrased softly.
4. Throughput collapses with context depth. 50.8 t/s at d=0, 21.5 t/s at 128k (~2.4x), 9.3 t/s at 256k (~5.5x). A single shallow tg128 number is not representative of long-context use.
5. The IQ1M build buys nothing in speed — 50.6 vs 50.8 tg128, i.e. no faster at all within run-to-run variance, for 7% less size, and it prefills 14% slower. Take it only if the 0.4 GiB matters.

8. Reproducing

python scripts/benchmark_variants.py     # pp512 / tg128, all variants
python scripts/eval_coherence.py         # executed-code coherence matrix
python scripts/measure_context.py        # KV + VRAM residency per context

Raw outputs: benchmark_variants_results.json, coherence_eval_results.json, context_scaling_results.json.