Forgotten-directive-Neon-stock
This is a merge of pre-trained language models created using mergekit.
An experimental merge to create a depraved model using Forgotten-Safeword-12B-v4.0 and Omega-Darker_The-Final-Directive-12B as a base.
- I merged Forgotten-Safeword-12B-v4.0 and Omega-Darker_The-Final-Directive-12B using nuslerp at 70/30 ratio.
- I created two derivative models using nearswap to boost already similar weights in Neona-12B
- I created another two models using arcee_fusion to add significant changes to Neona-12B
- I used model_stock to combine these changes - adding both most similar and most distinct changes, hopefully creating an interesting mix.
Oh, and I am planning to use this model as layer range for next KansenSakura update
Disclaimer: this is a VERY NSFW model, use at your own risk.
Merge Details
Merge Method
This model was merged using the Model Stock merge method using ./retokenized_FD as a base.
Models Merged
The following models were included in the merge:
- ./neona_fd_fusion
- ./forgotten-directive-neon-20
- ./neona_fd_fusion2
- ./forgotten-directive-neon-10
Configuration
The following YAML configuration was used to produce this model:
merge_method: nuslerp
models:
- model: ReadyArt/Forgotten-Safeword-12B-v4.0
parameters:
weight: 0.7
- model: ReadyArt/Omega-Darker_The-Final-Directive-12B
parameters:
weight: 0.3
dtype: bfloat16
merge_method: nearswap
base_model: kyx0r/Neona-12B
models:
- model: ./retokenized_FD
parameters:
t: 0.00010
dtype: bfloat16
tokenizer_source: kyx0r/Neona-12B
merge_method: nearswap
base_model: kyx0r/Neona-12B
models:
- model: ./retokenized_FD
parameters:
t: 0.00020
dtype: bfloat16
tokenizer_source: kyx0r/Neona-12B
merge_method: arcee_fusion
base_model: kyx0r/Neona-12B
models:
- model: kyx0r/Neona-12B
- model: ./retokenized_FD
dtype: bfloat16
out_dtype: bfloat16
merge_method: model_stock
base_model: ./retokenized_FD
models:
- model: ./forgotten-directive-neon-10
- model: ./forgotten-directive-neon-20
- model: ./neona_fd_fusion
- model: ./neona_fd_fusion2
parameters:
normalize: false
dtype: bfloat16
tokenizer_source: kyx0r/Neona-12B