Model Details
Atlantis 12B is a finetune of Mistral-Nemo-2407-Instruct in an attempt to make it a more RP/storytelling friendly model.
The model should support up to 128K context, though realistically in an RP setting I'd expect it to lose coherence above 24K.
This is a very early-stage and brand new finetune, so I expect it to have issues. Please read through the 'Usage' section before reporting any issues as Nemo has some specific requirements regarding backends, prompting formats & sampler settings.
Feedback
I appreciate all feedback on any of my models, you can use:
- My Discord server - requires Discord.
- The Community tab - requires HF login.
- The SillyTavern Discord thread - must be on SillyTavern Discord.
- Discord DMs to invisietch.
Your feedback is how I improve these models for future versions.
Formats
- FP16 Safetensors
- Static GGUF
- iMatrix GGUF (thanks to mradermacher)
- 4BPW EXL2 (thanks to Statuo)
- 6BPW EXL2 (thanks to Hydrafig)
- 8BPW EXL2 (thanks to Statuo)
Disclaimer
This model is built on top of Mistral's Nemo-2407-Instruct model and released under the Apache 2.0 license.
This model is fairly unaligned and can generate explicit, disturbing or offensive responses.
Use responsibly. I am not responsible for your use of this model.
Usage
Inferencing
If you are running a GGUF quantization, you need to make sure that you are running:
- llama.cpp b3436 or newer
- koboldcpp 1.71 or newer
- oobabooga 1.11 or newer
If the GGUF refuses to load, it is probably because your software is out of date.
Prompt Format
I would suggest using the Mistral prompt format:
[INST]Please write me a song.[/INST]
Other prompt formats may or may not work, but I've had most success with this.
Recommended Sampler Settings
Note the very low temperature, this is the same as Mistral's recommendation for the base model.
I'd recommend starting with:
- Temp: 0.3-0.5
- Top-P: 1
- Min-P: 0.05
- Rep Pen: 1.13-1.16
- Response Length: 350+
The model does prefer a longer response length.
Example Storywriting
These prompts are used on SillyTavern with a fairly basic narrator card. I have trimmed the start and finish where the narrator decided to add chapter headings, commentary and the like. All samples are made with the Q8_0 GGUF loaded with koboldcpp, with response length capped at 4096 tokens. Continue was pressed a maximum of once if the model cut off halfway through a chapter.
Training
Training was on a group of public & private datasets, using 2x A100 80GB GPUs.
In no particular order, I'd like to thank these people for their work compiling the datasets I used:
I also used a number of internal datasets compiled by myself on public & private data.