[!CAUTION] ⚠️ Warning: This model can produce narratives and RP that contain violent and graphic erotic content. Adjust your system prompt accordingly, and use Llama 3 chat template.
Meme-Trix MoE 14B A8B v1
A custom built Llama 3.1 8B MoE (Mixture of Experts) merge which combines Morpheus v1 with Assistant Pepe. The merge is suprisingly intelligent, detailed, and based. Scores ~15K at Q0 Bench. Fully uncensored and almost as fast as 8B dense. It also appears to have strong context retrieval ability. Asked for a summary at 16K and it works flawlessly (did not test higher yet).
If you want to merge custom Llama MoE you can add these scripts to your mergekit environment:
- mergekit-main\mergekit\architecture\moe_defs.py
- mergekit-main\mergekit\moe_init_.py
- mergekit-main\mergekit\moe\llama.py
Then assign the num_experts_per_tok in config.json (or the config.yaml)
Recommended Settings
(bolded kobold non-defaults)
- Temp 1.0
- TopNSigma 1.25
- Min-P 0.1
- Repetition Penalty 1.08
- Top-P 1.0
- Top-K 100
- Top-A 0
- Typical Sampling 1
- Tail-Free Sampling 1
- Presence Penalty 0
- Sampler Seed -1
- Rp.Range 360
- Rp.Slope 0.7
- Smoothing Factor 0
- Smoothing Curve 1
- DynaTemp 0
- Mirostat Mode OFF ("2" enhances creativity but also errors)
- Mirostat Tau 5
- Mirostat Eta 0.1
- DRY Multiplier 0.8
- DRY Base 1.75
- DRY A.Len 2
- DRY L.Len 320
- XTC Threshold 0.1
- XTC Probability 0.08 (The "Anti-Cliche" Shield)
- DynaTemp ON (The "Poor Man's Fading Mirostat")
- Minimum Temperature 0.65
- Maximum Temperature 1.35
- Temperature 1.0
- DynaTemp-Range 0.35
- DynaTemp-Exponent 1