[!CAUTION] ⚠️ Warning: This model can produce narratives and RP that contain violent and graphic erotic content. Adjust your system prompt accordingly, and use Llama 3 chat template.
Meme-Trix MoE 14B A8B v2
A custom built Llama 3.1 8B MoE (Mixture of Experts) merge which combines Morpheus v2 with Assistant Pepe. This should be superior to v1 and produce more intelligent responses.
If you want to merge custom Llama MoE you can add these scripts to your mergekit environment:
- mergekit-main\mergekit\architecture\moe_defs.py
- mergekit-main\mergekit\moe_init_.py
- mergekit-main\mergekit\moe\llama.py
Then assign the num_experts_per_tok in config.json (or the config.yaml)
Recommended Settings
(bolded kobold non-defaults)
- Temp 1.0
- TopNSigma 1.25
- Min-P 0.1
- Repetition Penalty 1.08
- Top-P 1.0
- Top-K 100
- Top-A 0
- Typical Sampling 1
- Tail-Free Sampling 1
- Presence Penalty 0
- Sampler Seed -1
- Rp.Range 360
- Rp.Slope 0.7
- Smoothing Factor 0
- Smoothing Curve 1
- DynaTemp 0
- Mirostat Mode OFF ("2" enhances creativity but also errors)
- Mirostat Tau 5
- Mirostat Eta 0.1
- DRY Multiplier 0.8
- DRY Base 1.75
- DRY A.Len 2
- DRY L.Len 320
- XTC Threshold 0.1
- XTC Probability 0.08 (The "Anti-Cliche" Shield)
- DynaTemp ON (The "Poor Man's Fading Mirostat")
- Minimum Temperature 0.65
- Maximum Temperature 1.35
- Temperature 1.0
- DynaTemp-Range 0.35
- DynaTemp-Exponent 1