merge
This is a merge of pre-trained language models created using mergekit.
Merge Details
Merge Method
This model was merged using the TIES merge method using meta-llama/Meta-Llama-3-8B-Instruct as a base.
Models Merged
The following models were included in the merge:
- ResplendentAI/Aura_Uncensored_l3_8B
- ResplendentAI/Kei_Llama3_8B
- Undi95/Llama-3-Unholy-8B
- vicgalle/Roleplay-Llama-3-8B
Configuration
The following YAML configuration was used to produce this model:
models:
- model: ResplendentAI/Aura_Uncensored_l3_8B
parameters:
density: 0.4
weight: 0.4
- model: ResplendentAI/Kei_Llama3_8B
parameters:
density: 0.4
weight: 0.4
- model: Undi95/Llama-3-Unholy-8B
parameters:
density: 0.3
weight: 0.2
- model: vicgalle/Roleplay-Llama-3-8B
parameters:
density: 0.3
weight: 0.3
merge_method: ties
base_model: meta-llama/Meta-Llama-3-8B-Instruct
parameters:
rescale: true
normalize: false
int8_mask: true
dtype: bfloat16
Open LLM Leaderboard Evaluation Results
Detailed results can be found here
| Metric | Value |
|---|---|
| Avg. | 67.88 |
| AI2 Reasoning Challenge (25-Shot) | 63.05 |
| HellaSwag (10-Shot) | 79.86 |
| MMLU (5-Shot) | 67.92 |
| TruthfulQA (0-shot) | 52.90 |
| Winogrande (5-shot) | 75.69 |
| GSM8k (5-shot) | 67.85 |