Embryophagus is a 12.5B model with 32K context length.
It is born from various merging experimentations backed by a homemade testing suite. I was lazy/not cautious and lost the exact recipe! Oops. However, I know its main DNA is from argilla/CapybaraHermes-2.5-Mistral-7B. I decided to share it because of its good results on common sense and logical tests. Just do not expect AGI, the moon or else. It DOES make mistakes. But less that many other RP oriented models that I tested and used... And I love the 32K context length, allowing generous character and/or world description...
Its main downside, like many other models, is to "play" sometimes as the user. It might describe a scene and suddenly writes the name of your character, and states what you are doing, impersonating you.
=> This is annoying but easily fixed by configuring custom stopping strings.
In text-gen-web-ui, go in "parameters>generation>custom stopping string", and insert there strings such as:
- "\nYourCharacterName:"
- "\nPlayer:"
- "\nUser:"
In my humble opinion, and this is very subjective, its English prose is beautiful. I like it so much that I will probably settle with this model for synthetic data generation.
Here are some examples:
"Your eyes fixate on the distant silhouette of "The Howling Cave" carved into the mountain's face. It appears as an enormous gaping wound, a jagged scar upon Mother Nature herself, where only darkness and death lurk within its depths. The villagers have spoken of horrors that dwell within; tales of ancient beasts, terrifying creatures and unspeakable horrors.
The path ahead is unclear, with twisted roots snaking upwards from the decaying forest floor like sinuous serpents reaching for their prey. Your senses heighten as your instincts take control - every rustling leaf, every crack of a branch echoes in harmony with your primal awareness."
[...]
"The sun's golden rays recede with every passing moment, yielding their place to an inky black sky littered with stars like diamonds scattered upon velvet cloth. The wind whispers cold promises against your skin as you push higher into the mountain's embrace, ever closer to the jaws of darkness that await within "the howling cave."
[...]
"With bated breath and a prayer on your lips, you slip stealthily into the cavern's dark embrace - skulking along its damp walls like a phantom of the night. The air within is cold and stagnant; thick with the scent of rot and decay that claws at your nostrils like sharp talons.
Your eyes adjust slowly to the darkness, illuminating just enough of the cavern's twisted depths for you to discern its horrors: skeletal remains littering the floor amidst pools of fetid water - their silent screams echoing in endless torment through this tomb-like lair.
As you inch further into the black abyss, your heartbeat pounds relentlessly against your ears - a drumbeat drowning out all other sound save for the hollow echoes of footsteps reverberating off the cavern walls. With every step, you sense an unnatural stillness creeping ever closer; like the breath of death itself breathing upon the back of your neck.
In this place where shadows dance and nightmares lurk, you tread with careful caution - a single misstep or errant whisper could spell certain doom for one who dares trespass within these ancient halls..."
For settings, I use usually text-gen-web-ui defaults
- temp 0.7
- top_p 0.9
- min_p 0
- top_k 20
- repetition_penalty 1.15
- etc.