Intent
The intent was to combine the excellent LemonadeRP-4.5.3 with WizardLM-2 in order to produce more effective uncensored content. While WizardLM-2 wouldn't balk at uncensored content, it would still falter in actually producing it whereas LemonadeRP didn't have this issue. The results are pretty good imo. There's a problem that if your response length is too long it will start to speak for the user but those usually disappear on swipes.
I had originally not intended to release this model and instead keep it private. It's my first foray into doing merges at all and I didn't want to release a subpar model. However, after encouragement I've decided to unprivate it. Hope you all get some enjoyment out of it.
Prompt - Alpaca
Using the Alpaca prompt seems to get good results.
Context Size - 8192
Haven't tested beyond this. Usual rule of thumb is that once you get up to 12k your responses tend to become less coherent and 16k is where things just devolve completely.
merge
This is a merge of pre-trained language models created using mergekit.
Merge Details
Merge Method
This model was merged using the SLERP merge method.
Models Merged
The following models were included in the merge:
Configuration
The following YAML configuration was used to produce this model:
models:
- model: Replete-AI/WizardLM-2-7b
- model: KatyTheCutie/LemonadeRP-4.5.3
merge_method: slerp
base_model: KatyTheCutie/LemonadeRP-4.5.3
dtype: bfloat16
parameters:
t: [0, 0.5, 1, 0.5, 0] # V shaped curve: Hermes for input & output, WizardMath in the middle layers