This is an abliterated version of the HALU 8B Llama3 v0.3 model.
GGUF: Static/Imatrix made available by mradermacher
The orthogonal abliteration process was performed on Kaggle's 2xT4 instance in under 30 minutes.
The orthogonal abliteration process used in this model is based on the method created by wassname, utilizing the Baukit library. The original code can be found in this GitHub Gist.
A slightly modified version of the earlier version of the original code was used, which aimed to improve readability. The notebook used for the abliteration process can be found here.
The following are the benchmark results from the Chaiverse Leaderboard.
The difference in safety scores is 0.10 between the standard version and the OAS version. This means the orthogonalization method works despite using very few examples.
WARNING
This model has not been extensively tested or evaluated, and its performance characteristics are currently unknown. It may generate harmful, biased, or inappropriate content. Please exercise caution and use it at your own risk and discretion.
NOTES
The model's temperature setting influences its refusal to generate certain content. Higher temperature values increase refusal, while lower temperatures reduce refusal.
Open LLM Leaderboard Evaluation Results
Detailed results can be found here
| Metric | Value |
|---|---|
| Avg. | 69.51 |
| AI2 Reasoning Challenge (25-Shot) | 64.08 |
| HellaSwag (10-Shot) | 83.35 |
| MMLU (5-Shot) | 67.80 |
| TruthfulQA (0-shot) | 53.45 |
| Winogrande (5-shot) | 79.79 |
| GSM8k (5-shot) | 68.61 |