Ornith-1.5-35B-A3B-BigBang-MTP
A TIES merge of ornith-ai/Ornith-1.5-35B-A3B (agentic-coding RL) and endless-frontier/BigBang-v1 (general tune), both post-trains of Qwen/Qwen3.6-35B-A3B — plus a working MTP speculative-decoding head, which stock Ornith-1.5 does not have.
The MTP finding
Ornith-1.5-35B-A3B ships 785 mtp.* tensors that are random initialisation, not trained weights: every projection has
std = 0.0200 with Gaussian kurtosis 3.0 (i.e. exactly initializer_range=0.02), and its norm weights sit near 0.02 instead
of ~1. Used as a speculative draft it accepts only ~13% of tokens (pure chance). This model replaces that placeholder with
the trained MTP head from Qwen3.6-35B-A3B, which transfers cleanly because Ornith's language tower is only ~0.2–1%
away from Qwen3.6 (measured cosine per tensor group).
Measured with llama.cpp speculative decoding (Q4_K_M main + Q8_0 draft, RTX 3090):
| draft head | acceptance | mean accepted run |
|---|---|---|
| Qwen3.6 grafted (this model) | 0.55–0.75 (code high, chat lower) | 3.2–4.0 tokens |
| Ornith-1.5 stock (random init) | ~0.13 | ~1.5 |
Merge recipe
- TIES (density 0.25, λ=1.0), computed in fp32 over the Qwen3.6-35B-A3B base, on all text weights.
- MoE router gates: kept verbatim from Ornith (never averaged — routing is where naive MoE merges break).
- Vision tower: kept verbatim (bit-identical between Ornith and Qwen3.6 anyway; this model keeps Qwen3.6's multimodal eyes).
mtp.*: Qwen3.6's trained head, verbatim (fused-expert layout, loads with the sameQwen3_5MoeForConditionalGenerationclass).
Evaluation (Q4_K_M, single RTX 3090, temp 0.1)
| this merge | stock Ornith-1.5 | |
|---|---|---|
| 15-task Python pass@1 | 15/15 | 15/15 |
| perplexity (mixed code+prose) | 3.34 | 3.41 |
| generation speed (no draft) | 131.0 tok/s | 133.9 tok/s |
| speed with MTP draft | 169.4 tok/s (+29%) | n/a (head is untrained) |
Chat, instruction-following, translation and creative prompts remain coherent (spot-checked; e.g. Welsh translation is understandable but slightly unnatural). This is a small local eval, not a benchmark suite — treat it as a no-regression check plus the MTP head-to-head, not a leaderboard claim.
Notes and caveats
- Reasoning is always on (
<think>), inherited from Ornith/Qwen3.6. Give generousmax_tokens. - Delta-interference probes (cosine between task vectors) showed Ornith's RL delta and BigBang's tune are near-orthogonal (cos ≈ +0.11), which is why the merge composes; this does not guarantee gains on any specific benchmark.
- GGUF quants + the MTP draft GGUF: EryriLabs/Ornith-1.5-35B-A3B-BigBang-MTP-GGUF
- Licences: Ornith-1.5 is MIT; BigBang-v1 and Qwen3.6-35B-A3B are Apache-2.0. This merge is released under MIT with attribution to all three parents.
Merged and measured by EryriLabs (Dwain Barnes), 2026-08-20.