Ministral-3-8B-Nymphaea-RP
A fine-tune of Ministral 3 8B Instruct 2512 for roleplay and creative writing.
A good choice for low-VRAM, I think. IMAO, better than Llama 3.3 8B Instruct.
[!Tip] The SillyTavern preset is available here. For custom presets, please use the Mistral V7-Tekken instruct template.
GGUF
Here is my custom mixed-quant GGUF, optimized for 8/12GB VRAM. If you need mmproj, it's available here.
llama-quantize \
--imatrix imatrix.gguf \
--token-embedding-type q8_0 \
--output-tensor-type q8_0 \
--tensor-type ".*attn_q.weight=q8_0" \
--tensor-type ".*attn_k.weight=q8_0" \
--tensor-type ".*attn_v.weight=q5_k" \
--tensor-type ".*attn_output.weight=q5_k" \
--tensor-type ".*ffn_up.weight=iq4_nl" \
--tensor-type ".*ffn_gate.weight=iq4_nl" \
--tensor-type ".*ffn_down.weight=q5_k" \
Ministral-3-8B-Nymphaea-RP.F16.gguf \
Ministral-3-8B-Nymphaea-RP.Q5_Mix.gguf \
q5_k
Imatrix file for making your own quants is available here. I used this calibration dataset to create it, expanding it with RP and creative writing data (about 400k tokens).
Training Notes
Trained on the latest iteration of my Darkmere dataset. This version features expanded genre variety, built upon a mix of manually curated synthetics and human-written stories.
[!IMPORTANT] The base weights are abliterated via Heretic prior to fine-tuning, so this fine-tune is quite uncensored.
Method:
- Training Method: DoRA (Weight-Decomposed LoRA)
- Target Modules
all-linear - LoRA Rank: 64
- LoRA Alpha: 64
- LoRA Dropout: 0.05
Hyperparameters:
- Batch Size: 2 (Per-device)
- Gradient Accumulation: 2
- Epochs: 2
- Learning Rate: 1e-4
- Optimizer:
adamw_torch_fused - LR Scheduler:
cosine - Noise Level:
neftune_noise_alpha=5
[!Note] The vision encoder was frozen during training, so the model retains its native vision capabilities.
Special Thanks
This fine-tune wouldn't be possible without the incredible work of the community:
- p-e-w for developing Heretic - an essential tool for censorship removal.
- SicariusSicariiStuff for developing SLOP_Detector script.
- Mistral AI for their Ministral 3 weights.
- AMD for their Instinct™ MI300X GPU.