Quantizations of https://huggingface.co/cgato/TheSpice-7b-FT-v0.3.1
Experiment
Quants ending in "_X" are experimental quants. These quants are the same as normal quants, but their token embedding weights are set to Q8_0 except for Q6_K and Q8_0 which are set to F16. The change will make these experimental quants larger but in theory, should result in improved performance.
List of experimental quants:
- Q2_K_X
- Q4_K_M_X
- Q5_K_M_X
- Q6_K_X
- Q8_0_X
Inference Clients/UIs
From original readme
The latest TheSpice, dipped in Mama Liz's LimaRP Oil. I've focused on making the model more flexible and provide a more unique experience. I'm still working on cleaning up my dataset, but I've shrunken it down a lot to focus on a "less is more" approach. This is ultimate a return to form of the way I used to train Thespis, with more of a focus on a small hand edited dataset.
Datasets Used
- Dolphin
- Ultrachat
- Capybara
- Augmental
- ToxicQA
- Yahoo Answers
- Airoboros 3.1
- LimaRP
Features ( Examples from 0.1.1 because I'm too lazy to take new screenshots. Its tested tho. )
Narration
If you request information on objects or characters in the scene, the model will narrate it to you. Most of the time, without moving the story forward.
Prompt Format: Chat ( The default Ooba template and Silly Tavern Template )
If you're using Ooba in verbose mode as a server, you can check if you're console is logging something that looks like this.
{System Prompt}
Username: {Input}
BotName: {Response}
Username: {Input}
BotName: {Response}