Notes on this Elarablated-v0.8 finetune:
This checkpoint was finetuned with a process I'm calling "Elarablation" (a portamenteau of "Elara", which is a name that shows up in AI-generated writing and RP all the time) and "ablation". The idea is to reduce the amount of repetitiveness and "slop" that the model exhibits. In addition to significantly reducing the occurrence of the name "Elara", I've also reduced other very common names that pop up in certain situations. I've also specifically attacked two phrases, "voice barely above a whisper" and "eyes glinted with mischief", which come up a lot less often now. Finally, I've convinced it that it can put a f-cking period after the word "said" because a lot of slop-ish phrases tend to come after "said,".
You can check out some of the more technical details in the overview on my github repo, here:
https://github.com/envy-ai/elarablate
My current focus has been on some of the absolute worst offending phrases in AI creative writing, but I plan to go after RP slop as well. If you run into any issues with this model (going off the rails, repeating tokens, etc), go to the community tab and post the context and parameters in a comment so I can look into it. Also, if you have any "slop" pet peeves, post the context of those as well and I can try to reduce/eliminate them in the next version.
The settings I've tested with are temperature at 0.7 and all other filters completely neutral. Other settings may lead to better or worse results.
Benchmarks:
Here are repeated phrase counts from before Elarablation:
...and after:
Obviously there's a lot more work to do (and since "slop" is somewhat subjective, it'll never be completely eliminated), but if you look at the frequency of repeated phrases, you can see that the numbers are noticeably lower in the "after" benchmark, which makes for a better writing experience.
Original readme follows:
~ We are Legion...
My biggest merge yet, consisting of a total of 20 specially curated models. My methodology in approaching this was to create 5 highly specialized models:
- A completely uncensored base
- A very intelligent model based on UGI, Willingness and NatInt scores on the UGI Leaderboard
- A highly descriptive writing model, specializing in creative and natural prose
- A RP model specially merged with fine-tuned models that use a lot of RP datasets
- The secret ingredient: A completely unhinged, uncensored final model
These five models went through a series of iterations until I got something I thought worked well and then combined them to make LEGION.
The full list of models used in this merge is below:
- TheDrummer/Fallen-Llama-3.3-R1-70B-v1
- Sao10K/Llama-3.3-70B-Vulpecula-r1
- Sao10K/L3-70B-Euryale-v2.1
- SicariusSicariiStuff/Negative_LLAMA_70B
- allura-org/Bigger-Body-70b
- Sao10K/70B-L3.3-mhnnn-x1
- Sao10K/L3.3-70B-Euryale-v2.3
- Doctor-Shotgun/L3.3-70B-Magnum-v4-SE
- Sao10K/L3.1-70B-Hanami-x1
- Sao10K/70B-L3.3-Cirrus-x1
- EVA-UNIT-01/EVA-LLaMA-3.33-70B-v0.1
- TheDrummer/Anubis-70B-v1
- ArliAI/Llama-3.3-70B-ArliAI-RPMax-v1.4
- LatitudeGames/Wayfarer-Large-70B-Llama-3.3
- NeverSleep/Lumimaid-v0.2-70B
- mlabonne/Hermes-3-Llama-3.1-70B-lorablated
- ReadyArt/Forgotten-Safeword-70B-3.6
- ReadyArt/Fallen-Abomination-70B-R1-v4.1
- ReadyArt/Fallen-Safeword-70B-R1-v4.1
- huihui-ai/Llama-3.3-70B-Instruct-abliterated
Recommended settings:
Temp 1.0
Min P 0.02
Because of the nature of this sort of 'Hyper Multi Model Merge', my recommendation is not to run this on anything lower than a Q5 quant.
If you enjoy my work, please consider supporting me, It helps me make more models like this! Support on KO-FI <3
This is a merge of pre-trained language models created using mergekit.
Merge Details
Merge Method
This model was merged using the DARE TIES merge method using TareksLab/L-BASE-V1 as a base.
Models Merged
The following models were included in the merge:
Configuration
The following YAML configuration was used to produce this model:
models:
- model: TareksLab/L2-MERGE2a
parameters:
weight: 0.20
density: 0.5
- model: TareksLab/L2-MERGE4
parameters:
weight: 0.20
density: 0.5
- model: TareksLab/L-BASE-V1
parameters:
weight: 0.20
density: 0.5
- model: TareksLab/L2-MERGE3
parameters:
weight: 0.20
density: 0.5
- model: TareksLab/L2-MERGE1
parameters:
weight: 0.20
density: 0.5
merge_method: dare_ties
base_model: TareksLab/L-BASE-V1
parameters:
normalize: false
out_dtype: bfloat16
chat_template: llama3
tokenizer:
source: base