My first merge and I honestly thought it would go worse, but I'm pleased with the results. Originally, this model was created to be merged with another model as a part of a personal project; now known as Blackfall Summanus. I wanted a 15B eRP model with the 'humanness' of Jamet-Blackroot with the chaos of Sunfall Stheno, tamed a little by some 'lewd' per se. It worked, mostly. By all means, this is an experimental merge, I'm sure there are problems I haven't found so please bear with me.
In limited testing, this model can't handle high temps at all. Anything higher then 1.3 will send it off the rails. Seems to play better with top P=0.8 and top K=25 on Llama3 template, but I'm still fooling with the settings. Has a tendency to ramble and can get quite dramatic, nothing too insane yet. Interently a little buggy and incoherent so proceed with caution.
L3-Blackened-Sunfall-15B
L3-Blackened-Sunfall-15B is a merge of the following models using LazyMergekit:
- crestf411/L3-8B-sunfall-v0.4-stheno-v3.2
- Hastagaras/Jamet-8B-L3-MK.V-Blackroot
- TheDrummer/Llama-3SOME-8B-v2
🧩 Configuration
dtype: bfloat16
merge_method: passthrough
slices:
- sources:
- layer_range: [0, 20]
model: crestf411/L3-8B-sunfall-v0.4-stheno-v3.2
- sources:
- layer_range: [6, 24]
model: Hastagaras/Jamet-8B-L3-MK.V-Blackroot
parameters:
scale:
- filter: o_proj
value: 0.0
- filter: down_proj
value: 0.0
- value: 1.0
- sources:
- layer_range: [8, 26]
model: Hastagaras/Jamet-8B-L3-MK.V-Blackroot
parameters:
scale:
- filter: o_proj
value: 0.0
- filter: down_proj
value: 0.0
- value: 1.0
- sources:
- layer_range: [24, 32]
model: TheDrummer/Llama-3SOME-8B-v2
💻 Usage
!pip install -qU transformers accelerate
from transformers import AutoTokenizer
import transformers
import torch
model = "kromeurus/L3-Blackened-Sunfall-15B"
messages = [{"role": "user", "content": "What is a large language model?"}]
tokenizer = AutoTokenizer.from_pretrained(model)
prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
pipeline = transformers.pipeline(
"text-generation",
model=model,
torch_dtype=torch.float16,
device_map="auto",
)
outputs = pipeline(prompt, max_new_tokens=256, do_sample=True, temperature=0.7, top_k=50, top_p=0.95)
print(outputs[0]["generated_text"])