So, despite creativity the first version had issues with tool calling and brains in long context role-play. I fixed this in V2 and changed recipe, putting more experiments on focus and filters.
Below is the exact mergekit_config.yml recipe used to synthesize this model:
Phase 1: Spark
Ignition of creativity with MeroMero V2 and Dark-Scarlett V2
merge_method: dare_ties
base_model: F:\AI\Merge\Gemma-4-it
tokenizer_source: base
dtype: bfloat16
parameters:
lambda: 1.0
models:
- model: F:\AI\Merge\Gemma-4-MeroMero-V2
parameters:
density: 0.50
weight:
- filter: "model.layers.4."
value: 0.60
- filter: "model.layers.9."
value: 0.60
- filter: "model.layers.14."
value: 0.60
- filter: "model.layers.19."
value: 0.60
- filter: "model.layers.24."
value: 0.60
- filter: "model.layers.29."
value: 0.60
- filter: "model.layers.34."
value: 0.60
- filter: "model.layers.39."
value: 0.50
- filter: "model.layers.44."
value: 0.50
- filter: "model.layers.49."
value: 0.40
- filter: "model.layers.54."
value: 0.40
- filter: "model.layers.59."
value: 0.40
- value: 0.50
- model: F:\AI\Merge\Gemma-4-Dark-Scarlett-V2
parameters:
density: 0.50
weight:
- filter: "model.layers.4."
value: 0.40
- filter: "model.layers.9."
value: 0.40
- filter: "model.layers.14."
value: 0.40
- filter: "model.layers.19."
value: 0.40
- filter: "model.layers.24."
value: 0.40
- filter: "model.layers.29."
value: 0.40
- filter: "model.layers.34."
value: 0.40
- filter: "model.layers.39."
value: 0.50
- filter: "model.layers.44."
value: 0.50
- filter: "model.layers.49."
value: 0.60
- filter: "model.layers.54."
value: 0.60
- filter: "model.layers.59."
value: 0.60
- value: 0.50
Phase 2: Form
Polisher of thoughts from Scotoma v2
merge_method: dare_ties
base_model: F:\AI\Merge\Gemma-4-it
tokenizer_source: base
dtype: bfloat16
parameters:
lambda: 1.0
models:
- model: F:\AI\Merge\Gemma-4-Scotoma-V2
parameters:
density: 0.60
weight:
- {filter: mlp, value: 0.2}
- {filter: self_attn, value: 0.2}
- {filter: q_proj, value: 0.5}
- {filter: k_proj, value: 0.5}
- {filter: v_proj, value: 0.6}
- {filter: o_proj, value: 0.8}
- value: [0.60, 0.60, 0.70, 0.80, 0.80]
- model: F:\AI\Merge\Dark-Mero-V3
parameters:
density: 0.40
weight:
- {filter: mlp, value: 0.8}
- {filter: self_attn, value: 0.8}
- {filter: q_proj, value: 0.5}
- {filter: k_proj, value: 0.5}
- {filter: v_proj, value: 0.4}
- {filter: o_proj, value: 0.2}
- value: [0.40, 0.40, 0.30, 0.20, 0.20]