[!IMPORTANT] ⚠️ Warning: This model can produce narratives and RP that contain violent and graphic erotic content. Although this model hasn't been ablated yet using Heretic / SOMPOA, refusals can be reduced by using
Gemma 4 26B NoThinkchat template in combination with Jailbreaks or Supervised Reward Preferencing (SRP).
This is a merge of pre-trained language models created using mergekit-exp, a custom fork of the official mergekit. This model was merged using the following (MoE-adapted) merge methods:
The following models were included in this merge:
- ConicCat/Gemma4-Writer-26BA4B
- Cyclone-Labs/Eclipsed-Phoenix-26B-A4B
- electroglyph/gemma4-26b-fiction-bf16
- google/gemma-4-26B-A4B-it
- Gryphe/Pantheon-Reasoning-26B-A4B-1.1-V2
- Hastagaras/G4-Jamet-26B-A4B-MK-1
- Naphula/Goetia-26B-A4B-v1.6
- Naphula/Promethean-Dawn-26B-A4B
- SubMaroon/Boulesis-v2.1-26B-A4B
- SubMaroon/Kitchoon-26B-A4B
- TheDrummer/Orion-26B-A4B-v1.1
- Vortex5/Shadow-Siren-26B-A4B
Thanks to the members of 26B-Suite for helping test and develop these models.
🙏 Thanks @mradermacher and other quantizers. 💾 GGUFs: - mradermacher/Goetia-26B-A4B-v1.7-GGUF- mradermacher/Goetia-26B-A4B-v1.7-i1-GGUF
⚙️ See this page for recommended settings.
🌋 Hellforged Parameters
mergekit_config_1_karcher.yml
architecture: Gemma4ForConditionalGeneration
merge_method: moe_karcher
# base_model: B:\26B\google--gemma-4-26B-A4B-it
models:
- model: B:/26B/ConicCat--Gemma4-Writer-26BA4B
parameters:
weight: 1.0
- model: B:/26B/Cyclone-Labs--Eclipsed-Phoenix-26B-A4B
parameters:
weight: 1.0
- model: B:/26B/electroglyph--gemma4-26b-fiction-bf16
parameters:
weight: 1.0
- model: B:/26B/Gryphe--Pantheon-Reasoning-26B-A4B-1.1-V2
parameters:
weight: 1.0
- model: B:/26B/Hastagaras--G4-Jamet-26B-A4B-MK-1
parameters:
weight: 1.0
- model: B:/26B/Naphula--Goetia-26B-A4B-v1.6
parameters:
weight: 1.0
- model: B:/26B/Naphula--Promethean-Dawn-26B-A4B
parameters:
weight: 1.0
- model: B:/26B/SubMaroon--Boulesis-v2.1-26B-A4B
parameters:
weight: 1.0
- model: B:/26B/TheDrummer--Orion-26B-A4B-v1.1
parameters:
weight: 1.0
- model: B:/26B/Vortex5--Shadow-Siren-26B-A4B
parameters:
weight: 1.0
parameters:
max_iter: 20
tol: 1.0e-9
router_strategy: karcher # Options: karcher, average, first, random_init
blend_experts: true # Blend corresponding experts (expert[0] + expert[0], etc.)
dtype: float32
out_dtype: bfloat16
tokenizer:
source: union
# chat_template: auto
trust_remote_code: true
name: 📜 Goetia 26B A4B v1.7A
mergekit_config_2_della.yml
architecture: Gemma4ForConditionalGeneration
base_model: B:/26B/google--gemma-4-26B-A4B-it
models:
- model: B:/26B/ConicCat--Gemma4-Writer-26BA4B
parameters:
weight:
- filter: "lm_head"
value: 0.1
- filter: "attn"
value: 0.1
- filter: "mlp"
value: 0.1
- filter: "embed_tokens"
value: 0.1
- value: 0.1
density: 0.9
epsilon: 0.09
- model: B:/26B/Cyclone-Labs--Eclipsed-Phoenix-26B-A4B
parameters:
weight:
- filter: "lm_head"
value: 0.1
- filter: "attn"
value: 0.1
- filter: "mlp"
value: 0.1
- filter: "embed_tokens"
value: 0.1
- value: 0.1
density: 0.9
epsilon: 0.09
- model: B:/26B/electroglyph--gemma4-26b-fiction-bf16
parameters:
weight:
- filter: "lm_head"
value: 0.1
- filter: "attn"
value: 0.1
- filter: "mlp"
value: 0.1
- filter: "embed_tokens"
value: 0.1
- value: 0.1
density: 0.9
epsilon: 0.09
- model: B:/26B/Gryphe--Pantheon-Reasoning-26B-A4B-1.1-V2
parameters:
weight:
- filter: "lm_head"
value: 0.1
- filter: "attn"
value: 0.1
- filter: "mlp"
value: 0.1
- filter: "embed_tokens"
value: 0.1
- value: 0.1
density: 0.9
epsilon: 0.09
- model: B:/26B/Hastagaras--G4-Jamet-26B-A4B-MK-1
parameters:
weight:
- filter: "lm_head"
value: 0.1
- filter: "attn"
value: 0.1
- filter: "mlp"
value: 0.1
- filter: "embed_tokens"
value: 0.1
- value: 0.1
density: 0.9
epsilon: 0.09
- model: B:/26B/Naphula--Goetia-26B-A4B-v1.6
parameters:
weight:
- filter: "lm_head"
value: 0.1
- filter: "attn"
value: 0.1
- filter: "mlp"
value: 0.1
- filter: "embed_tokens"
value: 0.1
- value: 0.1
density: 0.9
epsilon: 0.09
- model: B:/26B/Naphula--Promethean-Dawn-26B-A4B
parameters:
weight:
- filter: "lm_head"
value: 0.1
- filter: "attn"
value: 0.1
- filter: "mlp"
value: 0.1
- filter: "embed_tokens"
value: 0.1
- value: 0.1
density: 0.9
epsilon: 0.09
- model: B:/26B/SubMaroon--Boulesis-v2.1-26B-A4B
parameters:
weight:
- filter: "lm_head"
value: 0.1
- filter: "attn"
value: 0.1
- filter: "mlp"
value: 0.1
- filter: "embed_tokens"
value: 0.1
- value: 0.1
density: 0.9
epsilon: 0.09
- model: B:/26B/TheDrummer--Orion-26B-A4B-v1.1
parameters:
weight:
- filter: "lm_head"
value: 0.1
- filter: "attn"
value: 0.1
- filter: "mlp"
value: 0.1
- filter: "embed_tokens"
value: 0.1
- value: 0.1
density: 0.9
epsilon: 0.09
- model: B:/26B/Vortex5--Shadow-Siren-26B-A4B
parameters:
weight:
- filter: "lm_head"
value: 0.1
- filter: "attn"
value: 0.1
- filter: "mlp"
value: 0.1
- filter: "embed_tokens"
value: 0.1
- value: 0.1
density: 0.9
epsilon: 0.09
merge_method: moe_della
parameters:
lambda: 1.0
# normalize: true # deprecated
normalize_weights: false
normalize_router: true
int8_mask: false
rescale: true
router_strategy: della # average # random_init
blend_experts: true
dtype: float32
out_dtype: bfloat16
tokenizer:
source: union
chat_template: auto
name: 📜 Goetia 26B A4B v1.7B
mergekit_config_3_multifusion.yml
architecture: Gemma4ForConditionalGeneration
merge_method: multi_fusion # v4
base_model: B:/26B/Goetia-26B-A4B-v1.7_karcher
models:
- model: B:/26B/ConicCat--Gemma4-Writer-26BA4B
parameters:
weight: 1.0
- model: B:/26B/Cyclone-Labs--Eclipsed-Phoenix-26B-A4B
parameters:
weight: 1.0
- model: B:/26B/electroglyph--gemma4-26b-fiction-bf16
parameters:
weight: 1.0
- model: B:/26B/Gryphe--Pantheon-Reasoning-26B-A4B-1.1-V2
parameters:
weight: 1.0
- model: B:/26B/Hastagaras--G4-Jamet-26B-A4B-MK-1
parameters:
weight: 1.0
- model: B:/26B/Naphula--Goetia-26B-A4B-v1.6
parameters:
weight: 1.0
- model: B:/26B/Naphula--Promethean-Dawn-26B-A4B
parameters:
weight: 1.0
- model: B:/26B/SubMaroon--Boulesis-v2.1-26B-A4B
parameters:
weight: 1.0
- model: B:/26B/TheDrummer--Orion-26B-A4B-v1.1
parameters:
weight: 1.0
- model: B:/26B/Vortex5--Shadow-Siren-26B-A4B
parameters:
weight: 1.0
- model: B:/26B/Goetia-26B-A4B-v1.7_della
parameters:
weight: 2.0
parameters:
importance_metric: "pareto_auto" # Evaluates forward simulation loss across metric blends # 'pareto_auto', 'delta_mag', 'kl_div', 'cosine_sim', 'variance_sce', 'fisher_grad'
mc_trials: 64 # Deep 64-trial Monte Carlo continuous Dirichlet Pareto optimization
power_of_two_scale: 8192 # High-resolution 8K sampling stride across tensors
tukey_fence: 1.5 # Strict outlier gate for high-definition novelty grafting
anti_monopoly: 1.0 # Unconstrained training retention (rewards genuine fine-tuning), lower values compress raw norm disparity across the donors (not recommended)
sce_inversion_mode: true # Inverts minority sign conflicts rather than deleting them
is_moe: true # Set to false for dense models
router_strategy: "metric" # 'metric' (uses router_metric), 'average', 'first', 'base', 'random_init'
router_metric: "kl_div" # Evaluates exact categorical routing distribution on gate matrices # 'kl_div', 'pareto_auto', 'delta_mag', 'cosine_sim'
blend_experts: true # Blend corresponding experts (expert[0] + expert[0], etc.)
dtype: float32
out_dtype: bfloat16
tokenizer:
source: union
# chat_template: auto
trust_remote_code: true
name: 📜 Goetia 26B A4B v1.7C
mergekit_config_4_nearswap.yml
architecture: Gemma4ForConditionalGeneration
merge_method: nearswap # v2
base_model: B:\26B\Goetia-26B-A4B-v1.7_multifusion
models:
- model: B:\26B\SubMaroon--Kitchoon-26B-A4B
parameters:
t:
- filter: "layers.10.self_attn.o_proj"
value: 0.0001
- filter: "layers.10.self_attn.v_proj"
value: 0.0001
- filter: "layers.11.self_attn.o_proj"
value: 0.0001
- filter: "layers.11.self_attn.v_proj"
value: 0.0001
- filter: "layers.12.self_attn.o_proj"
value: 0.0001
- filter: "layers.12.self_attn.v_proj"
value: 0.0001
- filter: "layers.13.self_attn.o_proj"
value: 0.0001
- filter: "layers.13.self_attn.v_proj"
value: 0.0001
- filter: "layers.14.self_attn.o_proj"
value: 0.0001
- filter: "layers.14.self_attn.v_proj"
value: 0.0001
- filter: "layers.15.self_attn.o_proj"
value: 0.0001
- filter: "layers.15.self_attn.v_proj"
value: 0.0001
- filter: "layers.16.self_attn.o_proj"
value: 0.0001
- filter: "layers.16.self_attn.v_proj"
value: 0.0001
- filter: "layers.17.self_attn.o_proj"
value: 0.0001
- filter: "layers.17.self_attn.v_proj"
value: 0.0001
- filter: "layers.18.self_attn.o_proj"
value: 0.0001
- filter: "layers.18.self_attn.v_proj"
value: 0.0001
- filter: "layers.19.self_attn.o_proj"
value: 0.0001
- filter: "layers.19.self_attn.v_proj"
value: 0.0001
- filter: "layers.20.self_attn.o_proj"
value: 0.0001
- filter: "layers.20.self_attn.v_proj"
value: 0.0001
- filter: "layers.21.self_attn.o_proj"
value: 0.0001
- filter: "layers.21.self_attn.v_proj"
value: 0.0001
- filter: "layers.22.self_attn.o_proj"
value: 0.0001
- filter: "layers.22.self_attn.v_proj"
value: 0.0001
- filter: "layers.23.self_attn.o_proj"
value: 0.0001
- filter: "layers.23.self_attn.v_proj"
value: 0.0001
- filter: "layers.24.self_attn.o_proj"
value: 0.0001
- filter: "layers.24.self_attn.v_proj"
value: 0.0001
- filter: "layers.25.self_attn.o_proj"
value: 0.0001
- filter: "layers.25.self_attn.v_proj"
value: 0.0001
- filter: "layers.26.self_attn.o_proj"
value: 0.0001
- filter: "layers.26.self_attn.v_proj"
value: 0.0001
- filter: "layers.27.self_attn.o_proj"
value: 0.0001
- filter: "layers.27.self_attn.v_proj"
value: 0.0001
- filter: "layers.28.self_attn.o_proj"
value: 0.0001
- filter: "layers.28.self_attn.v_proj"
value: 0.0001
- filter: "layers.29.self_attn.o_proj"
value: 0.0001
- filter: "layers.29.self_attn.v_proj"
value: 0.0001
- filter: "lm_head"
value: 0.0001
- value: 0.0
parameters:
router_strategy: "nearswap"
dtype: float32
out_dtype: bfloat16
tokenizer:
source: union
chat_template: auto
trust_remote_code: true
name: 📜 Goetia 26B A4B v1.7D
🕵️ Pareto Telemetry Audit (1383)
| Category | Wins | Percentage |
|---|---|---|
| delta_mag | 583 | 42% |
| variance_sce | 233 | 17% |
| cosine_sim | 227 | 16% |
| fisher_grad | 219 | 16% |
| kl_div | 121 | 9% |


🖼️ Ars Galleria