NeoHorse-1-9B — MERNIK Quantization
MERNIK ("the one who measures") quants of TokenRhythm/NeoHorse-1-9B: imatrix-driven priority-queue allocation, judged by all three columns in synergy — PPL (canary), KLD-vs-ref (rank), scored tasks (verdict) — never by one alone. That synergy is the method; any single metric crowns its own artifacts.
Note: the Q8 / F16 / Q4 / Q2 tags in file names are for your convenience (build character hints) — pick by size, verify by the table below. All files here are MERNIK quants.
🧪 MERNIK protocol. This card leads with the verdict column; every build below carries all three — PPL (canary), KLD-vs-ref (rank), scored tasks (verdict) — published win or lose. Method + protocol:
wepiqx/MERNIK.
Results (all builds, all columns)
HE = HumanEval: 164 hand-written Python tasks, model writes code, hidden tests run it. pass@1 = solved on the first try — strict instruction-following plus real code, no partial credit.
Official BF16 posts 98.17% (context length undisclosed in the vendor
report — our runs below are -c 8192, so treat the gap as approximate).
llama-server --jinja, temp 1.0 / top_p 0.95 / top_k 20 / presence 0.0,
max_tokens 2048, fixed seeds. PPL/KLD: llama-perplexity -ngl 99 -c 1024 -n 64 -b 512 --seed 7, wikitext-2-raw, KLD vs Q8-proxy (no BF16 runs).
| Build | Size | PPL | KLD vs Q8 (proxy, not BF16) | pass@1 |
|---|---|---|---|---|
NeoHorse-1-9B-MERNIK-6500-Q8.gguf | 6.83 GB | 7.7695 | 0.0453 | 85.98% (141/164) 👑 |
NeoHorse-1-9B-MERNIK-6500-F16.gguf 🔭 | 6.83 GB | 7.8184 | 0.0505 | 82.32% — F16 attention core, built for distance (see below) |
| Q6_K (stock) | 7.36 GB | 7.9419 | 0.0118 | 82.32% (135/164) |
| TD-MSE (tested, not uploaded) | 6.83 GB | 7.7701 | 0.0452 | 82.32% — direction doesn't move distribution, moves code |
| TDN-MSE norms-shield (tested) | 6.83 GB | 7.7701 | — | 83.54% — shield recovers +1.2pp |
NeoHorse-1-9B-MERNIK-5100-Q4.gguf | ~5.1 GB | 7.8012 | 0.0586 | 82.93% — Q6-class code at −2.2 GB 👑, a machine: strict instructions, working code |
| Q4_K_M (stock) | 5.62 GB | 7.7824 | 0.0875 | 76.83% (126/164) — clean rerun; our pair beats stock both ways |
| MERNIK-5100-MSE | ~5.1 GB | 7.7962 | 0.0588 | 79.88% (131/164) — clean rerun; law holds by 3pp |
| Q2_K (stock) | 3.83 GB | 90.0284 💀 | 2.6354 💀 | 0.00% — stub completions, genuine collapse |
NeoHorse-1-9B-MERNIK-3650-Q2.gguf | 3.65 GB | 10.3586 | 0.4706 | 23.78% (39/164) — beats stock Q2_K (0%) at the same weight: at this size distribution decides everything |
The budget law: SMAPE owns small budgets (+3pp at 5100), MSE owns big
ones (+3.7pp at 6500). Same PPL both times, opposite code. Full analysis:
wepiqx/MERNIK.
⚡ Fast ring: GPQA-recognition (non-standard). Same builds scored by first-token letter likelihood (no generation, ~3 min): Neo-MSE-6500 51.5%, Neo-SMAPE-5100 48.0%. Rank agrees with HE here (MSE > SMAPE) but punishes thinkers — see MERNIK for the method and its limits.
🗺️ Capability map (field-tested, no mercy). 6500 writes full working games from scratch. 5100-SMAPE writes a complete working snake, no stuttering. 3650-Q2 writes correct syntax and surprisingly good Russian, but a whole game file is beyond it — short functions yes, projects no. Every weight has its jobs; the table above tells which.
🔮 Possible next: smape_frag. SMAPE-5100 already matches stock Q6_K (+1 task) at −2.2 GB. The fragility-modulated version could take the outright crown — but its fragility table costs serious compute. Want the table flipped? Say so in the discussions — community demand decides.
What is FRAG? Plain SMAPE aims the budget by theory (relative error). FRAG first measures every tensor group: each one is briefly dropped a tier and the real damage is recorded — hundreds of test builds. Groups that break easily ("fragile") get boosted, tough ones ("oak") get discounted. Theory aims, measurement corrects. That's why it wins — and why it costs.
⚠️ Sampling note. The vendor-recommended
presence_penalty 1.5breaks thinking models on this harness (HTTP 500peg-native format— the penalty corrupts the thought template and the model stalls). All runs here usepresence_penalty 0.0, verified clean across 3×164 tasks.⚠️ Context note. These runs used
-c 8192/max_tokens 2048, which is NOT enough for a thinking model: long chains of thought trip on the generation ceiling and the quant can't fully open up. At 16384+ the gap above should widen — this table is the floor, not the ceiling. Re-runs at larger budgets are planned and will be logged here.
Usage
llama-server -m NeoHorse-1-9B-MERNIK-6500-Q8.gguf \
-c 16384 --jinja -ngl 99 \
--temp 0.6 --top-p 0.95 --top-k 20
📝 Sampling for code: temp 0.6, top_k 20, top_p 0.95, min_p 0 — verified in the field (a full working snake game, Russian UI, almost no thinking needed). Benchmarks run at temp 1.0 per vendor protocol; real work wants 0.6.
🧪 Long context — help test. The main build keeps attention at Q8 almost everywhere (45×Q8 + 5×Q6 of 56); the TD-SMAPE file above holds a 949 MiB F16 attention core (qkv layers 20–30 + 3) — built for distance. Neither PPL nor KLD sees past short chunks, so we have not measured this. If you run needle-in-haystack or long-doc tasks, report your context length and score — real user numbers go on this card.
Quantization and tuning by wepiqx. Built with llama.cpp.
⏳ 40+ GPU-hours on a GTX 1070. Every number on this card was measured, not estimated — eleven HumanEval batteries plus a full PPL/KLD grid on an 8 GB card from 2016. If these quants serve you, a like or a test report is the fuel for the next round.