I had a lot of fresh and creative experience with both ConicCat tunes, but they had some instability issues, that I hopefully fixed there.
Phase 1:
merge_method: della_linear
base_model: F:\AI\Merge\Gemma-4-it
tokenizer_source: base
dtype: bfloat16
parameters:
lambda: 1.0
epsilon: 0.05
normalize: true
models:
- model: F:\AI\Merge\Gemma-4-Writer-D
parameters:
density: 0.30
weight:
- {filter: mlp.gate_proj, value: 0.25}
- {filter: mlp.up_proj, value: 0.25}
- {filter: mlp.down_proj, value: 0.25}
- {value: 0.40}
- model: F:\AI\Merge\Gemma-4-Writer-F
parameters:
density: 0.40
weight:
- {filter: mlp.gate_proj, value: 0.30}
- {filter: mlp.up_proj, value: 0.30}
- {filter: mlp.down_proj, value: 0.25}
- {value: 0.40}
- model: F:\AI\Merge\Gemma-4-Scotoma-V2
parameters:
density: 0.30
weight:
- {filter: mlp.gate_proj, value: 0.20}
- {filter: mlp.up_proj, value: 0.20}
- {filter: mlp.down_proj, value: 0.40}
- {value: 0.20}