Novelist end up to be raw and experimental. So I crank it up with more models, infusing creativity and worked on stability. Also added lm_head on top with StyleTune.
Below is the exact mergekit_config.yml recipe used to synthesize this model:
Phase 1: The Prose (dare_ties)
I took Novelist idea about description focus and remade it, so at the end less slop survive.
merge_method: dare_ties
base_model: F:\AI\Merge\Gemma-4-it
tokenizer_source: union
parameters:
lambda: 1.0
dtype: bfloat16
models:
- model: F:\AI\Merge\G4-Gutenberg
parameters:
density: [0.50, 0.50, 0.50, 0.40, 0.45]
weight: [0.40, 0.40, 0.40, 0.40, 0.40]
- model: F:\AI\Merge\Melinoe
parameters:
density: [0.30, 0.30, 0.30, 0.30, 0.35]
weight: [0.30, 0.30, 0.30, 0.20, 0.40]
- model: F:\AI\Merge\Glimmer
parameters:
density: [0.20, 0.20, 0.20, 0.30, 0.20]
weight: [0.30, 0.30, 0.30, 0.40, 0.20]
Phase 2: The Lead (model_stock)
The first stage was fine, but lacks of consistency, so I glued it with using two smart models.
models:
- model: F:\AI\Merge\NovelistX
- model: F:\AI\Merge\GarnetV2
- model: F:\AI\Merge\Gemopus
merge_method: model_stock
base_model: F:\AI\Merge\Gemma-4-it
dtype: bfloat16
tokenizer_source: base