Qwen3-4B-Nymphaea-RP
A fine-tune of Qwen3-4B-Instruct-2507 for roleplay and creative writing.
Suitable for mobile roleplay: tested on Nothing Phone 2 in Q4_K_M quantization (7-8 t/s)
[!Tip] The SillyTavern preset is available here. For custom presets, please use the ChatML instruct template.
Tested at Q8_0 quantization.
Training Notes
Trained on the latest iteration of my Darkmere dataset. This version features expanded genre variety, built upon a mix of manually curated synthetics and human-written stories.
[!IMPORTANT] The base weights are abliterated via Heretic prior to fine-tuning, so this fine-tune is quite uncensored.
Method:
- Training Method: DoRA (Weight-Decomposed LoRA)
- Target Modules
all-linear - LoRA Rank: 32
- LoRA Alpha: 32
- LoRA Dropout: 0.05
Hyperparameters:
- Batch Size: 2 (Per-device)
- Gradient Accumulation: 2
- Epochs: 2
- Learning Rate: 1e-4
- Optimizer:
adamw_torch_fused - LR Scheduler:
cosine - Noise Level:
neftune_noise_alpha=5
Special Thanks
This fine-tune wouldn't be possible without the incredible work of the community:
- p-e-w for developing Heretic - an essential tool for censorship removal.
- SicariusSicariiStuff for developing SLOP_Detector script.
- Qwen Team for their Qwen3 weights.
- AMD for their Instinct™ MI300X GPU.