Umbra-MoE-4x10.7
GGUF Files Kindly provided By @Kquant03
Umbra is an off shoot of the [Lumosia Series] with a Focus in General Knowledge and RP/ERP
This model was built around the idea someone wanted a General Assiatant that could also tell Stories/RP/ERP when wanted.
This is a very experimental model. It's a combination MoE of Solar models, the models selected are personal favorites.
base context is 4k but it stays coherent up to 16k
Please let me know how the model works for you.
Update:
A Umbra Personality tavern card has been added to the files.
Template:
### System:
### USER:{prompt}
### Assistant:
Settings:
Temp: 1.0
min-p: 0.02-0.1
Evals:
- Avg: 73.4
- ARC: 70.31
- HellaSwag: 87.81
- MMLU: 66.42
- T-QA: 67.82
- Winogrande: 83.27
- GSM8K: 64.75
Examples:
User:
Tell me an interesting fact.
Umbra:
The Amazon Rainforest is home to an astounding variety of life forms.
It encompasses over five million square kilometers across nine South American countries and holds approximately one-third of all known species on Earth.
This biodiversity hotspot supports more than 390 billion individual trees representing 16,000 species.
Over two million insect species live here alongside thousands of bird, mammal, amphibian, and reptile species. In this dense forest, indigenous communities have developed intricate knowledge systems that allow them to thrive sustainably within their surroundings.
User:
Tell me a story about a red haired fox
Umbra:
In the enchanted forest of Ardenwood, a vibrant world teeming with life and magic, lived a curious creature known as Vixie, whose russet fur glowed against the verdure that surrounded her. Her emerald eyes twinkled with mischief and wonder as she roamed the lush landscape, befriending all who crossed her path.
One sunlit afternoon, Vixie found herself weaving amongst a sea of tall grasses swaying gently in the breeze. Ahead lay a shimmering riverbank, where a gathering of animals was assembled around a mystifying sight - a radiant stone pulsing with an ethereal glow.
A majestic stag approached Vixie, his voice resonating with authority. He introduced himself as Elthon, leader of this assembly. They had gathered to seek guidance from the celestial gem, said to hold the secrets of the universe within its sparkling depths.
Vixie, filled with curiosity, asked if she could partake in their quest for knowledge. Elthon agreed, and they each took turns gazing into the jewel. As Vixie's turn came, she felt a tingling sensation course through her body, her senses heightened as the stone revealed a vision before her.
In the dreamlike tableau, Vixie saw herself standing atop a hill overlooking Ardenwood. She watched as the seasons changed, the land thrived, and creatures played out their destinies. It appeared the fates of these beings were intertwined, a tapestry woven together by love, friendship, sacrifice, and growth.
When the trance subsided, Vixie returned to the present moment, feeling humbled and grateful. This revelation emboldened her desire to cherish those bonds she shared with others, knowing how vital they were to the beauty and harmony of the enchanted realm. From then on, Vixie continued her adventures with renewed purpose, nurturing friendships and embracing the mysteries of Ardenwood.
Umbra-MoE-4x10.7 is a Mixure of Experts (MoE) made with the following models using:
- kodonho/SolarM-SakuraSolar-SLERP
- Sao10K/Sensualize-Solar-10.7B
- NousResearch/Nous-Hermes-2-SOLAR-10.7B
- fblgit/UNA-SOLAR-10.7B-Instruct-v1.0
🧩 Configuration
base_model: kodonho/SolarM-SakuraSolar-SLERP
gate_mode: hidden
dtype: bfloat16
experts:
- source_model: kodonho/SolarM-SakuraSolar-SLERP
positive_prompts:
- "versatile"
- "helpful"
- "factual"
- "integrated"
- "adaptive"
- "comprehensive"
- "balanced"
negative_prompts:
- "specialized"
- "narrow"
- "focused"
- "limited"
- "specific"
- source_model: Sao10K/Sensualize-Solar-10.7B
positive_prompts:
- "creative"
- "chat"
- "discuss"
- "culture"
- "world"
- "expressive"
- "detailed"
- "imaginative"
- "engaging"
negative_prompts:
- "sorry"
- "cannot"
- "factual"
- "concise"
- "straightforward"
- "objective"
- "dry"
- source_model: NousResearch/Nous-Hermes-2-SOLAR-10.7B
positive_prompts:
- "analytical"
- "accurate"
- "logical"
- "knowledgeable"
- "precise"
- "calculate"
- "compute"
- "solve"
- "work"
- "python"
- "javascript"
- "programming"
- "algorithm"
- "tell me"
- "assistant"
negative_prompts:
- "creative"
- "abstract"
- "imaginative"
- "artistic"
- "emotional"
- "mistake"
- "inaccurate"
- source_model: fblgit/UNA-SOLAR-10.7B-Instruct-v1.0
positive_prompts:
- "instructive"
- "clear"
- "directive"
- "helpful"
- "informative"
negative_prompts:
- "exploratory"
- "open-ended"
- "narrative"
- "speculative"
- "artistic"
💻 Usage
!pip install -qU transformers bitsandbytes accelerate
from transformers import AutoTokenizer
import transformers
import torch
model = "Steelskull/Umbra-MoE-4x10.7"
tokenizer = AutoTokenizer.from_pretrained(model)
pipeline = transformers.pipeline(
"text-generation",
model=model,
model_kwargs={"torch_dtype": torch.float16, "load_in_4bit": True},
)
messages = [{"role": "user", "content": "Explain what a Mixture of Experts is in less than 100 words."}]
prompt = pipeline.tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
outputs = pipeline(prompt, max_new_tokens=256, do_sample=True, temperature=0.7, top_k=50, top_p=0.95)
print(outputs[0]["generated_text"])